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ABSTRACT 

A primary goal of this proi^ct was to develop an 
interface that would provide direct access for inexperienced users to 
existing online bibliographic information retrieval networks. The 
experiment tested the concept of a virtual-system mode of apcess to a 
network of heterogeneous interactive retrieval systems and databases. 
An experimental translating computer interface named CONIT,.' that 
enables the virtual-system mode, was developed as a research tost 
vehicle. The interface was designed ^o- make the basic functions of 
three different bibliographic retrieval systems easy to u:?'e, even by 
inexperienced end useirs, by providing a simplified common command 
language coupled .*rit h ^extensive online instruction. Analysis of \ 
cgntrolled experd^'ents with end users indicates t,he probable success 
of operational interfaces using the virtual-system principle and 
other technigues Jdemonstrated in the experimental interface. The 
research has also suggested that certain techniques impl^mentable on 
an interface could' enhance retrieval effectiveness for a wide class 
of users by aiding them in the development of seacch strategies. A 
prelect bibliography and references -are included. (Author/PAA) 
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ABSTRACT 



Investigations have been continued and concluded on the concept 
of* a virtual -system \node of access to a network of heterogeneous 
interactive retrieval systems and databases: An experimental trans- 
lating computer interface, named CONTT, that ^enables the virtual -system 
mode, was developed as a research test vehicle. The interface was 
designed to make the basic fimctions of three different bibliographic 
retrieval systems easy to use, even by inexperienced end users, by 
providing a simplified common -command language coupled with extensive 
online instruction. Analysis of controlled expoCiments with end users 
indicates the probable success of operational interfaces using t!ie 
virtual- system principle and other techniques demonstrated in.^the 
experimental interface. The research has also suggested that certain-' 
techniques implementable on an interface could enhance retrieval 
effectiveness fo;r a wide class of users by aiding users in the devel- 
opment of search strategies. • 
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1.1 Overview 

. t « 

The research reported ilpon here considered inetliods of improved utiliza- 
tion of existing online bibliographic information retrieval systems with heter- 
ogeneous char^icteristics . A primary goal of the resoarcli was to investigate 
the technique of interposing a computer interface between inexperienced end 
users and the retrieval' systems. - This interface would permit end users to 
access the systems themselves, directly, rather than through professional inter 
mediaries, as is generally^ required now. 

•The concept investigated consists of an interface, implemented as a 
separate computer system whicli interconnects the heterogeneous systems into a 
network through which information is readily retrieved by hiding ' system dif- 
ferences and providing other user aids. To test the validity of this concept, . 
there was developed an experimental translational interface that mapped many 
features and protoc9ls of three bibliographic systems into a single, common 
or "virtual'' system. ' In the primary operating mode of the interface, it is 
this virtual system with which end Users interact. A "pass through" mode 
is also provided, however, so that a. user with experience in any .one system 
in the network can engage that system in its own language. 

. The three systems included in the network were ORRIT (Systems Development 
Corporation) , DIALOG (Lockheed), and MEDLINE as implemented at two locations: 
the National Library of Medicine (NLM) and the State University of New York 
(SUNY) . Collectively, tjiese systems provide^ access to approximately 120 data- 
bases and some 60 million documents. 

An experimental interface designated CONIT, 'standing for Connector for 
Networked' Itiformatioii Transfer, was implemented on the M.I.T. MULTICS computer 
system and evaluated in a series of controlled experiments , involving six en^ 
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search strategiei^' was part of the experimental work. 



users. A comparative study of online arjd offline techniques for formulating 
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1 . 2 The Import anco or (]ott .hij^ jind JJsjt>rs Onl i no 

It is iinportant to ^et the end user online for soveral reasons. Many end 
users prefer to do thojr own searching; they wish to avoid the hassle of try- 
ing to make their needs understandable to someone else. rh(jy also wish to 
avoid the requirement of making an appointment with an intermediary searcher 
which may mean an inconvenience in time, location, or both. If the end user 
is not present during searchang, the relevance- feedback benefits of winter- 
active se^irching are lost. Finally, making it easy for the end user to do 
his own searching has a compounded potential for reducing costs: the cost of 
an infomation specialist can be eliminated or ^t least greatly reduced, thus 
making online searching more attractive to end usei\s; and the increased usage 
of retrieval thus Created has rk)tential for increasing the use factor of 
vendoi's* systems, thereby offering, at least in principle, the , opportunity 
for further user-cost reductions through the economies of the factor of scale. 

1 , 3 Barriers to Ejld-JJser Searching 

. ^ r 
Ease of use was alleged to be an inherent feature of online retrieval 

systems when they were first introduced. It was proposed that, through a 
simple interactive dialog with the computer, any user would be able to get 
the information he or sjhe needed quickly and easily. Howev-er, because early 
, designers lacked the experience needed to appreciate the difficulty of build- 
ing into interactive-computer systems the necessary human-factors features 
that would enable the systems to be easily used,^ and because of the prolifera- 
tion of systemj, each with its owi) access requirements, it is professional 
intermediaries acting on behalf of end users, rather than the end users them- 
selves, who are actually retrieving the information being sought. Even the 
intermediaries regularly require a training period of up to a week for each ^ 
new system that must be learned. Furthermo^p, as Benenf eld (1975)* has 

« reported, inteinnediaries must. spend a considerable part of each work week 
practicing searching in order hot to '^lose touch*', as well as spend a number 

^ of hours per week trying to keep up with system changes and new databases ' 
Specialized characteristics of infrequently used databases are easily forgotten^ 

Wanger (1976) has quantified the extent to, which ei)d users (i.e., those who 



* RejFe'^ences are listed alphabetically by author and date in 
Section 6. 
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ult imately need and use the i n formal i on ]*ot !'U>ved) operate (ho systems 
themselves. She reports' that an extensive study of' I'et r i eviU -sy s ( em use 
showed that the eiul ii^er perTormed fii.s own scareh only 7.3 peix^ent of tlie 
time. Others who havC reported on problems of end users of interactive sys- 
tems include Mann (197^), Curtis ( 1977), and Kennedy (197S), In fjict , (lie 
notion has been expressed in recent information science conference, discus- 
sioTKs that there really is no ho]>e of bririgiiig the end user online. Our 
research on this point, however, leads us to the conclusion that is is 
entirely feasible to do so provided certain aids to simplify u^se can be 
incorporated into the computer systems. ''^ 

1,4 riie r 1' a n s 1 a t i. n g - 1 n t e r f a c e / V i 1 1 1 a 1 - Sy s t em Ap|^i'o ac h / 

In order to investigate means to surmount ■ obstac/les hi.iidering conven- 
ient aud effective use of the multiplicity of heterogeneous interactive bib- 
liographic retrieval systems^ the M.l.T. Laboratory for Infonnation and 
Decision Systems undertook a research program to examine the feasibility of 
interconnecting interactive retrieval systems through computer intferfnces. 
The computer- interface is intended to achieve compatibility among systems' 
of heterogeneous hardware and software components through use of common 
retrieval prot'bcols, or b>< translating dissimilar protocols to a common 
set (see Fig . 1) . • 

Y . 

Our research progr?wi emphasized an approach in which ^the interface, is, 
in effect, a common system into Wliich, and .^^^jn which, requests and results 
are translated automatically as they flow between user and serving systems. 
This approach has the virtue that a usex^<<^t tempting '^'to retrieve information, ^ 

f k K 

whqn entering through the access mechanism provi^ded by the commoti ipterface, 
sees a single virtual system in whiclr all the complexities of the different 
retrieval systems and databases are hidden; only a single uniform system is 
apparent. In this way, the goal of convenient use of heterogeneous coiflputer 
resources is achieved, at least for the particular application of interactive 

bibliographic retrieval systems. 

j -• - •» • ^ ■ . ^ 

There are four aspects of our approacfi which, taken together, distinguish 
ouT^ efforts to achieve networking: (1) we concentrated on the inft)rmatioiV 
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Pig. 1. Logical Diagrjim of Virtual -System Interface. 



tiaiisfor application, wttl^ particular cMnpl\a,^is on online b i b ! i oj^MapIn c 

retrieval systems; (2) we iitili/.cHi exjst.iMj.;, major, stand-alone interactive 
^ ^ '- - '* 

sy St ems w i t hoii t mod 1 f i ca t :i on ; {'S) we plriced emphasis on seivinj; the ordinal^' 
end user that Is^ the user exj>ei .i enced neither \\\ computer proynamm i , 
j^oneral computer ustige, nor in the use of interact' ive retrieval systems; 
and (4) we replaced existinj^, heterogeneous, often d i ft' i cu 1 1 - to-use 
coHiputer/human .ifUerfaces with a simpler, common .easier- 1 o-us.e interface. 

Oiiv research on this project has included investigat ions of the 
following related areas: 

1) Networking approaches for heterogeneous systems 

2) Human factors and instructional considerations for oiUine systems 

3) Retrieval functions and common languages 

4) Indexing conversion and other searcli technj c'jues 

5) Commonality among database structures 

6) I.ogical and software structures for Tietworked interfaces * 
V 7) lixperimental interface systems 

.8) User evaluation with an experimental interface 

9) Cost /benefits tradeoffs - 

Our previous work in these areas has been detailed in reports* and papers 
(see Section 6: Project Bibliography). 

^ ' ^ O^^tline of Current Work 

In the remainder of this report, we describe the work that has been/ per- 
formed during the period of this grant. May 1977 through April 1979. In 
Section 2, we desflji^ibe the development of an enhanced experimental interface 
that has .been used as a vehicle for investigating the virtual-system/ trans- 
lating-computer-interf ace concept. This new development on the CONIT inter- 
face has included several aspects: 

1) The addition of several retrieval functions to the virtual mode 

2) Improvements to the instructional dialog 

3) The incorporation of a^ftill-f ledged virtual system mode in which 

tlve user may bypass system selection and go directly to data- 
base selection \ 

* ' ■ ■ . . ; \ ' 

u 



^1) h^foiMnat ion at Xhc interface onab.liiij; users to make 
intoVined jiKlyinent s on dat abase se lect ion 

Jn sect/ion 3, we describe a series of experime!\ts run wit!) actnaj 
e!\d users of CONIT to test the effectiveness of tlie vaiious interface 
techniques wo have einj)loyod. lif feet i veiu^ss is measured, in terms ol the 
facility with which users learn the CON IT coimnands and the quality of 
the retrieved results. 

/ Sectionj^ 4 and 5 contain an overall analysis of t^ie iiUerf ace con- 

cept And its potential in various contexts. 

. C/ 

Sections 6 aiul"? contain a project b.ibliograpl^y and references 
for this report, respectively, « ' • . 

The ap]>cndices contain detailed listings of instructiojial material 
and experimental usagps. 
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2. liNHANCIil) CONIT r^XPI'RTMliNTA!, TNTIiRl-AC!-. 

Tn a previous report '(P 8*), we described i\ Versioii of CONTT, 
which we could call '^CONTT 2'\ on which experiments with end usore wore' ^ 
performed. Results of these experiments indicated that end users could, 
'mdeeU, be brought online*, but that there 'were ,a number of areas in* , . , 
which interface capabilities needed to be enhanced before effective use 
by inexperienced users was possible.' Since then, a nevy version of CONI T, 
which we qall ''CONIT 3'*, has been created which incorporates several 
enhancements to CON IT 2. In this section we first sumjii^rize thfcse enhance- 
ments, next wgi^giye an overall description of CONIT 3 through *^an example 
session; and finally, we summarize the commands in CONIT 3. 

2.1 New CONIT Features 

2.1.1 Improvem ents^ to User Ins^ruc^tioijs 

Experience with CONIT 2 suggested improvements to the instruc- 
tional dialog which have been implemented. These included additions to, 
and modifications of, messages given by CONIT. Some of these messages are 
explicitly requested by the user by the EXPLAIN (HELP) conmiand, and others 
are, given by CONI.T as part of the regular instruction in' a given context 
[e.g., what to do after the response to the FIND (search) command is given]. 
Attention was given to providing online instruction when it would be most 
useful, and inhibiting ovelrly repetitious messages. Thus, for example, a 
special message was added after the second search reminding the user of . 
the COMBINE coHim^nd. Another special message after the first (but no other) 
user error in giving a command name reminded the user about how to correct 
typiiig error3. The full list of requestable (EXPLAI^) messages for CONIT 3 
is given in Appendix A. A partial accounting of the context dependent mes- 
sages is given explicitly in the sample user dialog- given in Section 2.2 

As will be explained in detail in Section 3, various instructional 
materials., in additidn to the online instruction, were prepared in printed 
form for offline use. These included* a printed version of the EXPLAIN 
messages (see Appendix A) plus^ special instructions ^for the development , . 



* "P-numbex'' refere^ices refer to documents in the 
Project Bibliography (Section 6). 



of oiiliiio searching straloj^ics nnd the ^ejection o\' tlatahasrs . 
2.1.2 New I'Uiiclions (The ''''^'f'^ ^ * '^^y "i^i"^!!]) 

CON IT. 2 pcrnullod most ot* \hv hnsic ret im oval funct ions t o'^bc 
requested in the common (X)N1T lauj^uiage. In (>ONrr , a lunnboi^ of'^acUl i t.i ona I 
(unctions were nrade expressible in the common Lanp^age. Tl^ese additional 
(Yn^ctions include; (1) author-immo searching (as well as >^ubject- term 
searching); (2) the ability to request offline output from the remote host 
retrieval system; Jind (3) the ability to ge>t information about a d^itabnse 
and/or select a database at any point in the online session, regardless 
of what, if any, retrieval system. is currently connected. 

This third added capability is extremely important, because it real- 
izQS .t}\c virtual- system concept. By that, we mean it is no longer neces- 
sary for the user to be concerned about the dif ferent^'retrieval systems , 

f " ' 

as such. She or he need be concerned only about jl^^^ databases to be 
searched. V 

This capability was provided by having information about each data- 
base in the CONIT interface. A user, then, can find out about available 
databases directly from CONIT. When a user selects. a database for search- 
ing^ CONIT checks to see what systems have that database. If the database 
is ^available on the currently connected system, than a simple database 
selection command is Issued in the command language of the current system, 
"otherwise,^ a system that d oes ^ have the given database is connected to 
automatically by CONIT (after automatically disconnecting the current 
retrieval system, if any>. Thus, the user can ignore thd issue of dif- 
ferent retrieval systems Wther with respect to which systems have which 
databases, or which system (if any) is currently connected. Of course, 
the differences among system command languages had already been submerged 
in. CONIT 2. System connections for C(WIT 3 are shown in Fig. 2. 

2.2 Example of CONIT^ Session 

In order to understand the experiments and analyses in subsequent 
sections of this report, tt--4*-^lpful to have an appreciation of the 
features and capabilities ,of the CONIT interface system. Perhaps the 
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bes t way to gain tins appreciation js to review the operation of CON IT 
for a typ i cal " onl i ne session, l-or this purpose, have chose?) t lu^ on- 
line session of one of Ihc CONl'l' users in our last roujul of experiments. 
1-xcerpts of a slightly motlifiecl version of (hi^ session are reproduced 
in Appendix H. The mod i fi ca t i ops include a ^few additions to the actual 

session, in order to illustrate certain foatiircs. The user commands 
V 

are labalod IJI, 1)2, .... The CONfT system responses Tire labeled C1,C2, 

. A reproduction of oi^c pag^ of the original typescript is shown 
in ligui^e 3. 

Prior to the s(:art of the sessioi^ itself, the CONli' .interface sys- 
tem has been logged in and set up at a terminal connected to the MULTICS 
computer. The user is told to start hi,s session by typing ^le word 
'*start*'. followed by a carriage return on the terminal (Ul) , CONlT's 
response (CI) is a message welcoming the user and giving him a simplified 
method for correcting typing errors and explaining how to get instruct- 
ional help. l\^hile help is s^iggested, the user is given the option of 
going off and doing what he wants (a general principle in CONIT) . 

1Vo aspects of the user/system interactive dialog are introdiiced in 
this first message: (1) the user should wait for the user t^ue whicli 
indicates that it is his turn to "talk", and (2) the user indicates the 
completion of his command by a qarriage return. Two other pointsj are 
illustrated by message CI: (1) suggestions on what to do next are given 
and highlighted by special format (separate line with five space indenta- 
tion contrasted with three-space indentation for paragraphing) ; and 
1*1 

(2) explanatory information is given in "churlks" with the size of the 
chunk being related to the context (messages early in the session are 
generally Shorter and cover fewer- facts than later ones) . 

T|ie user follows the system advice and asks for help (U2) . In its 
resptm^e (C2) , CONIT introduces the. EXPLAIN command. In doing so, CONIT 
also introduces through example the idea of the command-name/argument 
format for the user command langauge, and the idea that abbreviations 
are permissible for shortening user commands. 
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In (.'3, the user stfos a list of the \>asic commniuis ;uul a short okplnn- 
at ion of (kkMi. TIhs a{iu>imt oi' dctnll wns ihvs i j^iuul \o ho suffUioiit to 
f^ivo the user an overviovv of the interface .system ca])ab i 1 i t i os aiui a ^;uido 
line for future instruction without overbu rdon in}^ hiin/lior with details. 
This goal is aided by having relatively few di fforoni batic commands. C3 
also gives additional instruction on the basic prbcediires for the inter- 
active dialog. Much of this instr\iction has boon .^oon by the user before; 
the technique of repeated instruction ^i s a purposeful approach in order 
to achieve learning reinforcement. Tn C3 the concopt of a hiera rch ical ■ 
structure to ii|Struction is brought fo!>/ard. The simple, general explan-^ 
ation is given first; more detailed exj)lanations are available, at user 
request or system d^Lscretion, to elaborate on various aspects of the 
initial explanation. The inexperienced user is prompted to get an explan- 
ation of the PICK command next; this follows the princi^ple of giving, or 

♦ 

suggesting, instrviction on a system feature at the first point in the 
session when the user is likely to need the feature x 

In C4, COIilT explains how the PICK command is used to pick database^ 
for searching. In acc'oixlance with the virtual- system approach, the user 
is' not required to consider what retrieval systems are appropriate. 
However, information is_ .given on how the,, explanation for system specifi-^ 
catdon,^^ich a sophisticated user, might want, can be requested. 

The response to th^ SHOW DATA command (C5) lists the seven subjett 
areas chosen as a high-level classification scheme for the databases to . 
which CONIT has access. Again, following the hierarchical approach pre- 

' * ^ . ^ 

sented by CONIT, the user specifies one^of these areas (Area 5) for 
detailed database listing in the command ^ SHOW DATA 5 (U6) . 

For each database, CONIT. gives a common CONIT name and number, the 
"standard" name^ a short explanation of the database, and an indication of 
which retrieval systems have the database. The CONIT name is intended to 
be niore indicative of the subject content of the database than the "stand- 
ard" name which is often a meaningless ' character string to the uninitiated 
(note that different systems sometimes give different names to the same 
database). Any of these ndtnes is allowed for use in the PICK command, as 



* The actual experimental user on which this sample session .is based 
did not issue commands U5 or U6, since he was m the group of users 
who were , presented offline printed explanations of databases (as 
explained below in Section 3), ■ I 



well as the (X)N.IT number, wliich provides a short <*orm tliat i iicorpoiat rs 
the (X)Nrr classi ficat ion scheme. 

Generally, the short explanation of a database is suTficient iiHor- 
mation upon which a user can de.cide whether to search the database. How- 
ever, additional infomiation is available through the liXPLAIN DATA name 

(. " ^ 

command. (Name is the name, or number, for the database on which infor- 
mation is desired.) By this comm^uul, the information about a database 
available online from a retrieval system is requested of that system. 
If the retrieval system which has the information is not currently connected 
CONTT connects to it (see bejow for discussion of automatic connection 
protocols). The information itself, is obtained by CONT'f's sending the 
VlLWn and ?FlliLl)n commands to DIALOG (n = the DIALOG number of the data-^ 
base) or tl>e ''KXPLATN namej' command to the !:LI1ILL* or ORB^T systems 
(name - database name as known by the host retrieval system) . ^ 

Next, the user picks the SOCIAL SCT12NCGS CITATION INDGX database 
(CONIT name SOCSCI, C'ONIT number SI), with the command PICK 51. CONIT 
recognizes that the database is available only on the DIALOG system (at 
tlie time of this experiment) and sets about connecting to DIALOG. Note' 
In this virtual system mode of operation the user need not be concerned 
about where the database actually is; CONIT will find it for him. If 
the database is available on more than one system, CONIT will choose 
according to the following algorithm: i 

V 



1) If there is a currently connected system and the database Is 
available on it, that one is chosen; 

23 Otherwise, the "normally preferred" system is cho;?en. (Prfef 
erence is now preset, although it could be dynamic, and 1 
based on such criteria as which system is more readily ava^l 
. aole or which database implementation is more complete or 



<^pstly. For "example, SUNY/MEDLINE is preferred over NLM/NfeDLINE 



less 



:ie 



because it is usually less busy, and for that reason, is t 
system usfers are urged to use by the National Library of Med- 
icine for accessing the .MEDLINE, SDlt^INE, or MESH vocabulary 



* ^EyilLL^ is the official name fdr the National Library of 
Medicine's retrieval system, which is often known by its 
database,' MEDLINE, ; 



largest 
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tiat al)ases . ) ^ 

When a database is i)ickcd, aiul no system is currently coiniecteJ, 
is the case in this example, CONIT must request the ^f^l!.Tl(:S system 
establish an appropriate physi cal network connection. This requires 
c placing oT a .telephone call wIvLch is accomplished by the automatic 
/dialing equipment. The physi/.al coRnection takes about 30 seconds 
during whicli time the messaj^es given in tjie Tirst nine lines iol lowing 
the ' ♦' + + ^CON IT: ' header in C7 are presented to the user. These mes- 
sages serve four purposes : - ' 

1) Feedback concerning CONlT's understanding of which database 
was pi'cked is given the user for checking purposes. 

2) The identification of the interconnecting network* (here TYMNIiT) 
and the system chosen is given; this information is not neces- 
sary ^ but may be useful -~ for example, in case system or 
network is • unavai lable. 

3) Additional instructional information is given' to, the user 
' concerning his interactive, dialog with the interface. 

4) The user is kept informed of what is'^^ going on,, and not left 
to wonder, because he sees nqfthing happen, if something 
has gone wrong. 

The fourth purpo5e provide^'^he main /reason for giviiig the remaining 
parts of the response C7. Each of these lines is given after some corrob 
orate(i response is obtained in the sequence of: (1) placing the telephone 
call; (2) identifying the terminal properly to the network and inhibit- 
ing network echoing of characters; (3) establishing- the connection to 
the host retrieXral system; (4) logging into the retrieval system^^^ with 
kppropriate identification and/or password strings; (5) in some systems, 
establishing the abbreviated version of the dialog (between CON\J and 
the retrieval system) , and, J^nal ly (6) connecting to the desired data- 
« base. The entire process -fsilj^omatic, and while messages are given 
the user to keep him assured that progress is continuing, h^ need not be 
concerned with any of 'the many, details of the connection process. One 
aspect of the host messages^ of the connection procjsss is fed back to the 

* The network is chosen' by tONIT if the user does not specify. The 
defaujt network was TYMNET here; at -other times' it has been TELENET. 
The preset choice of network is b^ised primary^^^onjvhldwietwork was 
easier to make local connections to in the recent past. 
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user for his poss ihle *iiit ores t : any broadcast nows message. No such 
message appeared in (his login, !>ut such news wa^; foci back \k\ login 
to ORBIT shown in CS6 below. 

The explanation of the FIND command (C8) gives the user an exposi- 
tion of the keyword/stem, free- vocabu I ary-based* i iii^ la 1 searcli strategy 
recommended by CONIT. The user then rtiquestcd explanations ((:9-C'14) of 
additional information on searching, as recommended by OONl'f. 

At this point in the session, a commiini cat ions failure occurred. 
The user picked the databasel^gain , and communications were re-established 
The user, who was searching on \\\o subj^ect ''world economic and political 
models'* then followed one of the previous CONIT suggestions anjl used the 
SHOW INDFX command to browse the index of the S^OCSCl database for terms 
alphabetically near twCjWords related to his search: "model" and 
"world". The SHOW INDKX CQinmand is translated as the KXPAND command 
in' DlAf.OG and the NlilGllBOR (?*mmand in liLHlLL and ORBI T. 

The^ user follows the CONIT suggestion to search on keyword stems 
with his fii'st search: FIND MODIBb. Following GONITIS searching prin- 
ciples of searching all possible indexes on stemmed forms, this is 
trmislated as SELFICT MODRL? -~ i,e., do a truncated search on the given 
word ill the basic index. In FIJIILL and ORBIT, this would be translated 
as "FIND ALL T^jlODEL: These latter two systems do specifically allow 
searching all tndexes,, in contrast to DIALOG, which requires a dif- ^ 
ferent kijid of I search request for each non-basic index. 

The host i^spoiise from tliis search takes longer than a preset figure 
(here set at f iv(!^^^^3^ec^ Wl^PevSr long delays like this occur, CONit 

gives the user the option to wait longer for the response, or' give up 
and go on to something else. This procedure is another manifestation 
of the doctrine of avoiding excessive time during which tli^e user might 
be unsure of What, if anything/ is happening or what he should do in 
the absence of response. Of .course, this pro|:edure also jiandles the case 
where, due to some unforeseen Occurrence, the host system neVer will 
respond • We may note that the detailed explanation on what to do in . 
this delay situation is presented to the user only in the first instance; 



in later instances, it is assumecl that the usor knows what to do. 

The power of the truncated or stemmed search is demonstrated by 
the fact that the results for set I and set ^! show significantly hi^'Jier 
retrieval counts than would untruncated searches. 

Note the special message at the end of the response to t!\c second 
■search, suggesting the combine operation; this follows the principle of 
providing su^gestiqns when most likely to be needed. -The user here had 
already seen the explanation on the COMBINU command aiid so goes ahead to 
perform^ it. 

The user next issues the SHOW command to see catalog output for 
retrieved documents from the current set. After getting more detailed 
information on |iOw to get more particular output information, the user 
requests all the information available on document 4- He then requests all 
on all documents in the "current set be output offline (at the remote host 
site) and mailed to M.l.T. 

Just a few interactions selected from the remainder of the session 
will be chosen to illustrate some other system features. In U33, the 
user has ma4e a syntactic error (left out spacp after COMBINE Command); 
he realizes this and f^trikes the BREAK key to "ViU" the line up to 
that point and return to CON IT command level in order to resobmit command 
in correct form. In U53, the. user employs the Boolean AND NOT operator . 
in a COMBINE command to eliminate documents already retrieved. In U56, 
he decides to search a new database SSIE (the Smithsonian Science In- 
formation Exchange database containing research project s;immaries) . At 
the time of the experiment, this database was available through CONIT 

♦ 

only on the ORBIT system. CONIT recognizes this fact and logs in to 
ORBIT after loggihg off of DIALOG. ' 

The user employs the samd CONIT commands to redo his search in 
SSIE; of course, different translations are involved. In Ue4, the SHOW 
REVIEW command is given to request a listing of all searches done on 
SSIE during the session until' that time. In U90 the session is terminated 

•■ ■ . ^22 .. . ■• 
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with the STOP command; CON-IT automaticaJ ly logs off of (he coimoctod 
retrieval system before tenninatinR the sessioii. 

2 . 7> Suinmary of CON IT CommaiKls ^ 

The functional capabilities of CONIT are .summarized in the follow-- 
ing list of coimnands:* y 

I * 

2.3.1 Instruct ion- Requesting 

EXPLAIN concept 

An explanation of the concept or item des;iRnated conce])t^ 
is given *» 

. • HELP = r-XPIJVIN EXPLAIN (Get-started message given) 

2.3.2 Datg ^base Information 

Show DATA- ' ' 

Lists seven areas of data bases . 

^SMOW DATA n 
Lists databases in area n 

liXPLAIN DATA, database [system] ^ 
Get the explanation of database ^l^tabase from system system 

/ - : " . 



2.3. 3 batabase Selection 



/ 
/ 



PICK databa se [system] [network] 
; ConiTiect to database dat abase oYi system system through network net-, 
work. First log crff currently connected system and select default system 
and database, as required; 

^ ■ - ■ . , ■ . • , 1 

2.3.4 Sear ch and Index Operations , 
FIND subject 

.' Do a truncated search in the basit index (and in other indexes, 
if possible) on the subject term(s) subject. 



* In the explanation of .conunands, we use underlining in examples of 
f (J lapniguage constiTuction to indicate variable elements and bracketing, 

IERIC ' to indicate optional' arguments. > 



Do a trvmcatod, search in the author index inuUvr iho. name av_L'->»;- 

O 

♦ 

SHOW INDliX terrn 

Pertonn t.he browsing coiiunand to display t.orins alphabetically near 
term from the basic index of the currently connected database. 

,SMOW Ri;VU!W 

All searches in the currently selected database are reviewed, 

COMBINl' SR T , i bool^ Sl'T k 

The sots designated i and are combined according to < the Boolean 
operator booj^, where bool = ''and", "or", or "and not". 

2.3.5 Outputting .Document In^oriimt^ion 

SHOW I i)ocuMf::N:rs i_ - k] [set s^] [type] [OFP] 

Informati6n of the type type for documents numbered through k of 
set s^ is given. The types allowable are "title", "abstract" , "citation 
(title, author, and source)", and "all"; the default type is "citation 
The default documents are the first' five. If k is missingf; . information 
on the single documeht i^ ,ls output. If OFF is present, the information 
is output offline; otherwise, online display is' assumed. The order in 

which the arguments appear is optional. , 

I • - ' 

2.3.6 Misce l laneous Com mands 

i 

SEND conunand * 

The command command is sent directly. to the connected remote host 
retrieval system without^' translation. 

START ' 

Begin, or restart, session. 

STOP . ; 

■ ■ ■ ' ' 

^ End session (disconnect any connected systems' first) 



DISCONNIiCT 

Discoiuioct coiHioctod. systems {h\\{ do not oiul S(^ssion) 
COMMr.NT <^oinincn^ 

The iisor^s coimncnl commont is savoJ by CONIT I'or fnturo rovicw by 
t he research pro j ect J nvest igat ors . 

SHOW STATtlS 

rhc names of. the currently connected database, system, network 
are dis|llayed. 

SHOW NIIWS 

The tiirrent news items from the connected retrieval systems are 
displayed to the user at the terminal. 

In addition to the commands meant for. regular use by ordinary end 
users, there are a number of commands designed for use by the CONIT system 
implementers and operators. These commands pe^^nit the fol low.i ng operations : 
selecting a rule table to guide the session and translations; making mod- 
ifications to the current rule table; and putting the CONIT system .in a 
mode to accommodate debugging operations. 

2.3.7 Interrupting and Uditing 

Tlve BRKAK key can be used to intei^nipt action of CONIT or 
the reipote system. If CONTT action is being interrupted, control, is 
returned immediately to the user for further commands. If remote-system 
action is being inteivrupted, then CONIT sends a simulated BREAK (series of 
nulls) to that systpm (if it accepts breaks); in .any case, CONIT then 
waits for a signal from the remote system s^^gnifying -that it is ready for 
additional commands before returning a user- cue to the CONIT user. 

The MUbTICS character-cancel code (//) and line -cancel code (0) are used. 
In addition, the BRBAK key can be used to kill the line^ and get a new 
user^ cue. - • - 



3. INTl.RlACr. I'.XPI'.RIMI.NTS 

ISA Object ivx)S 

The bixxul objective of the cxper inioiit ;i 1 pro}.7rain was to ovnluntc 
the CONIT iiUerface in tcruis of its ability to allow end us^rs to satis- 
fy their informational needs by accessing a network of. heteroj;joneous 
infonnatlon systems tfvrough the interface. Several measures of effect- 
iveness were considered: quantity of j nf oi'nia ti on recovered; usefulness 
af recoyored in format Ion; effectiveness of individual searchps ( as 
measured by precision) and of tho overall session (as measured by recall); 
the time requii'fed to arrive at various stages of the search; and general 
u^ei' satisfaction witli the interface. 

A second objective was to make a comparative analysis of an online 
and offline approach to search-strategy formulation.' As discussed in detail 
below, we have come to the conclusion that an end user's ability to formu- . 
late a good search strategy is crucial to his success in extracting useful 
information from the network. Trained information specialists have con- 
siderable sk:^ll .in amiving at a gocxl strategy; how to substitute for 
this skill when an inexperienced user is online was one segment of our 
investigations. , 

. ""^ 

3.;2 The ■ Exp erimental Users a nd Their Infonnation Needs 

We sought typical end users> to serve as experimental users (ElPs) of 
,the. CONIT interface system. Such users were solicited through notices 
posted at locations scattered around the M.I.T. campus. These notices 
offered free information from computer databases in return for participa- 
tion in our experimen]ts. Responders to the offer were interviewed to 
determine their information needs /and backgrounds, especially with respect 
to computer experience. Basically/ the six l^U^s were the first i*espoiiders 
who could be fitted in with the experi. mental schedule. Several res- 
ponders were tuilned kway, either because they were already proficient \ 
searchers in one of more computer retrieval systems, and/or they did not ; 



have an immediate, bona fide need in tcM'ms of a well del' i nod ( op i c of 
current personal interest. 

rhe general search topics and ])rofossional backgi'ounds of the six 
chosen to be EU^s are given in Table 1* All six liU's had used computers 
online to a limited extent, but not for infonnation retrieval. However, 
four of the six had observed others do online bibliographic searching. 



EU No, 



Table 1 . General Search Topics of the Six liU's 

General" S earch To pic 



Professional 
Level 



Graduate student in 
Library Science; part- 
time secretary 

Engineer; part-time 
gr^uate student in 
Aeronautics 

Graduate student in Mech- 
anical Engineering and in 
P51itical Science 

, * Graduate student in 
Physics 

Research Staff member; 
Post-doctoral Fellow 

Graduate student .with 
background in Chemistry, 
Library Scieirce and 
Social Science 



The interface between municipal 
librarieis and social service agencies 



Airplane wing section design 



World models in economics and 
politics 



Cross sections of helium ions under 
electron bombardment 

Measurements of twe-ear phenomena 
in persons with a hearing loss . 

Oral exj^ninations as a testing tech- 
nique and" -curriculum design in 
medical education* 



None of the EU*s had been using computers regularly in the recent past. 
One of them was ^n excellent typist, four were fair, and the remaining one 
classified himself as a^'hunt and peck" typ^^. All of the EU*s were in an 
academic and /or research environment. Five of the six were currently study- 
ing at the graduate level: two fi^llT-time M.I.,T. doctoral candidates, one- . 
special (part-time) M.I.T. graduate stbd^nt, and two Master's degree candi- 
dates at neighboring colleges. The sixth„ EU was a post-doctoral Fellow. - 



p 



The soarcli topici^ covored a witlc variety of subject areas, i iic liul i uj.'^ 
thi^eo in social science areas aiul throe in pl^ysical, jnetlical, ov cu^i^w- 
ocring sciences. # 

3 . 3 r.xperimental Vy oceduv e s ^ ^ 

All experiments were carefully controlled and monitored. To the 
extent possible, the same set of procedures was used for each r.U and the 
same set of data points was recorded. Three F.U ' s (T-Ul, l:U2, and 1^U3) ^ 
identified as- ^'Group A'', began the development of their search strategies 
off line;'^l.n accordance with a set of printed instructions given to them 
at the outset of the session before they engaged the computer terminal. 
These instructions concern database selection and search-strategy form- 4t ( 

Illation; they are reprcxluced in Appendix C. - 

The other three P.U's WS and l'U6)-, identified as "Group B", 

did not get the offline instructions; they were assisted solely by the 
online instructions given through the terminal. In other words, all CU's 
were given online instruction through CONIT, while Just the first three 
had additional offline Instruction. 

At the .beginning of each experimental session, an experimental super- 
visor (ES) briefed th^ EU concerning the nature of the experiment. In 
order to keep this briefing as nearly uniform as possible, a common brief- 
ing statement (see Appendix C) was read to^ the EU. The oral briefing 
took from 3.5 minutes to 6.0 minutes, and averaged 4.6 minutes in length 
for the six EU's.. 

Following the briefing, the Group A EU's were given the offline 
instructions. These users spent between 20 and 36 minutes with these 
materials,vas shown in Table 2. The >experimental supervisor answered 
the EU's questions after the debriefing and during their use of tlie off- 
-Irine instructions. The online session itself — i.e.,. the EU's inter- 
action with CONIT at the computer terminal - began immVliately after 
the briefing for the Group B EU'S and immediately .after the offline 
preparation' mentioned above for the Group A EU's. The experimental super- 
visor remained in the room where the EU worked at the terminal. The purpose 



of this arr<ingcmont w?is to jn-ovide i!iu)!)t.i;Us i ve monitorinj^ ol tlie 
session and oT the l:U (beyond what was pos'iiibjc from the full record 
of user coinmands and CONT'l' responses, as maijjtained in a computer 
''audit*' file). The supervisor, j^at at a tcrti^|nal located at the opposite 
end of the room from the one at'^which the \-X\ worked. At the terminal, 
the supet*t;?i sor feigned work unf<jylnted to the JIU, but actually used the 
terminal to monitor the TONTT/FU interactions. The* supervisor could 
also observe visually any activity of the Ml) not recorded on the computer. 
The CU himself was asked to make written or mental notes of particular 
problems or other reactions he might have, as to be able to relate 
his/her experiences more completely to the supervisor at a post-session 
dcibriefing. The si^rvisor did not prompt, the HU or otherwise interfere 
with the course of the interaction unless computer system bugs or computer 
c6mmunications problems appeared to hinder the continuation of the session; 
If few such problems did cr*op up, as explained below. 

The online session was terminated when the l-U issued the S'fOP com- 
mand. The prim/i^y motivation f oi^'tterminating the session appeared to be 
the CUVs feeling that he was at the point of diminishing returns, as far 
as getting useful infonnat*lon' was concerned . At the end of the online 
session, the supervisor held a debriefing conference with the KU, as a 
means of getting his reactions while they were fresh in his mind. This 
debriefing included a detailed review of the session interact ions using 
the terminal t>^pescript and any note of the El) or supervisor as guides. 
The review emphasized: 1) any problems encountered by the El); (2) the 

.... .s 

rationaljip for EU search strategy formulation; and (3) relevance judgments . 
by the fcU dn retrieved documents. The EU's were asked to judge relevance 
on a foUr-ppint scale: high, medium, low, and none. ; 

Based^on- the relevance' judgments given by'the EUs during the experi- 
mental' sessions, we subsequently performed an analysis of search strategies 
that might improve oh the searches of the l#)'s. Searches based on this 
analysis, bpth in databases pickGd^by the El) and in others, were then 
performed. Catalog Output from selected documents ret rieve^t' from these 



r 

so»irches, alonj? with \Uv ol'{'\'\i\c oiitp\it rcqurstrd duriiij.^, tlic online 
experimental session, was tlien presented to tlie !iU i'ov t'urtluM* relovance 
judgments. In a couple of cases, a second round of analyst searches 
and 1-U relevance judgmoiU.s ,was cai^ried out before a final analysis of the 
experiment was completed. " ^ 

■ 

3*4 Hxj)e r im e n t a 1 Results 

3.4.1 (^lant ities Measured 

Statistical data derived from the various stages of the exper- 
imental analyses are summarized in fable 2. The measurements for the ind~ 
ividual EU's'are given along witli averages for eacli quantity measured. 
Averages are shown for: (1) Group^A, the of fline-instruction, users (l-Ul-3); 

(2) Group B, the users having online instruction only (nU4-6); and (3) all 

■m 

six lilPs (FJjl-6) taken as a single group. 

The quantities measured include the times required to reacli various 
cxitical junctures of the session and numbers related to how busy the 
CU was and how many useful documents were retrieved, compared to the 
number potentially retrievable in the databases: The times measured in 
(2a) through (2d) refer to the time from the beginning of the online ses- 
sion at the terminal. The time-to-f irst-PICK (2a) is how long it took the 
EU to pick adatabase in' which to 'searcJu The time-to-f irst-FIND (2b) is the 
time until the first search conmiand was issued. The time-to-f irst-SHOW (2c) 
is the time until the first output^ on retrieved document references was 
requested. /Fhe time-to-first-useful-reference is how long it took before 
the EU saw a document reference that was useful to his or her problem. 

The number of commands issued online by the EU (4a) and the resulting 
number of online typescript pages (4b) each page is about 60 lines, some 
of' which are blank (see Fig. 3 for typical page) are measurements of 
♦•busyness*'. These figures are normalized , for length of time at terminal v_. 
in (5a) and (5b) . 

The number of document references requested by the Eli (8) is broken 
down to^those shown online (8a). and those requested* for offline output (8b). 
.Similarly, the number of documents judged useful by the EU (9) based on 
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38 


90 


72 


114 


M. 


64 


89 


77 








1 1 




19^ 


30 


27 


16 


25 


Z 1 


5. 


Rate of interaction 






















5a • Commands /minute 




0.7 


0.8 


1.0 


0.9 


0 . 8 


0 . 9 


0 . y 


1 0.9 




5b, Pages/hour 


17 


' 12 


13 


.17 


15 


1 b 


14 


16 


1 ^ 
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6. 


Total session time (Min) 


70 


70 


146 


69' 


124 


108 


98 


100 


7. 


Number of: 






A 








/ 








7a, Databases, searched 


2 


1 


2 


2 


2 


2 


1.7 


2 


1.8 




7b. Retr. systems used 


2 


1 


2 


1 


2 


2 


1.7 ■ 


1.7 


1 17 


8. 


Number doc references : 
















/ 


1 37 




8a • shown online 


20 


7 


68 


29 


60 


38 


32 






8b. printed offline 


40 


50 


{319 


d 


0 


0 


136 


0 


1 

1 OO 


' 9. 


Number useful refs: 






















9a. seen online 


4 


3 


34 


. 1 


31 


Q 

O 


14 




1 1^ 

1 ( 




9b. total found by user 


19 


30 


171 


1 


31 


8 


11 


13 


45 


10. 


Recall base, est.- 


960 


1000 


7:30 


140 


1200 


670 


890 


670 


780 


Ml. 


Recal 1 


.02 


.03 ' 


.23 


.01 


.03 


.01 


.09 


.02 


1 .05 
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the particulnr catalog output sooi^-- is iiulicatoc! Tor oiiTmr rcMr.irva! 

(<)a> separately from tl\e total fouiKl (9b). I'rccision fiyMiri^*; are implicit 

in those statistics, although the precision so derived is Tor sclecled 
povtions of selected searches. 

The recall base was estimated on the basis oT relevance Judgments 
inad^ by the TAJ on i^ainples of documents retrieved in post-session analyst 
searches, i^or purpose of this summary, a document was considerd ^Vrelevant^^ 
:if it was marked as being of ^'high'' or ''moderate'' relevance by the HI); 
'4ow"-relevance documents are lumped with ^'no''-relevance documents as being 
nonrelevant. For the purposes of the six I^U's, relevance was closely 
related to usefulness, although we shall qualify this point later. The 
recall base was derived by a simple extrapolation on the s^amples shown to 
st.he bU: as will be discussed below^ this is simply a lower bound on the 
Actual recall base that would be found through fully exhaustive search 
methods • 

3.4.2 Overview of Results^ 

The results generally show that the EU's were able to use CONIT 
and find a miinber of relevant dociunents on their own. However, because 
particular circumstances affect the results so strongly, we need to con- 
sider more details concerning the individual sessions in order to properly 
assess the degree of effectiveness of various aspects of the interface. 
A summary of pertinent session details is given in Appendix D. 

In attempting to evalute the effectiveness of the experimental CONIT 
system -- and thereby, the potential effectiveness of similar interface 
systems -- we might start by considering one measured quantity in Table -2: 
the online time until, the first useful reference was retrieved (2d). The 
average o^ 'this measurement over all six EU's was 35 minutee. This figure 
can.be judged in either a ^ne^tive or a positive light. On the one hand, 
if the EU had worked with arr information specialist intermediary who was 
experienced in the several systems involved, the same results could undoubtedly 
have been achieved more quickly. 

On the other hand, the experimental results suggest that inexperienced 



users ca!), with the aid of iiUorface tcchn icjuos/ fiad i ni oi^nat i oi^ iVcVin 
hot orogonoous systems and databases in reasoaablc pcrioiis o( time. Note 
that what is iiu^ludod within this time j^erVx! is learning how to inter- 
act with CONir at the terminal, and specifically, learning and using 
coimnands to get explanations, pick databases, perfoi^m searches, and display 
information on r.etrieved documents. In' addition, for the lUJ's /)f Group B 
(and partially for those of- Group A), information luul to be alAorbod 
regarding the nature of particular databases and the techniques for . 

^ 

search-Strategy fonnulation. 

In making more det^i evaluations of the interface and its potential 

it is worthwhile to consider two related but separable aspects of the RU's 
use of the system: (1) the mastery of the CONIT command language and 
modes of interaction, and (2) the adequacy of the development of search 
strategies. lioweV^^tr,^ before looking at these two aspects in detail , we 
first take up the f|uestion of supervisor's assistance. 

Since there were a number of instances of supervisor interaction with 
the EU during the online session (see Appendix D) , it is appropriate to con 
sider the extent to wliich the results were altered by this deviation from 
the desired goal of'^no human interference. Odr opinion is that while some 
documents might not have been retrieved, or might have taken longer to 
retrieve, without supervisor aid, the basic conclusions about potential 
interface effectiveness will not be affected. jjr 

The justification for this opinion is based on the observation' that 
there were three main kinds Oif situations in which the supervisor inter- ^ . 
vened. In the first- type' of situation, there^ vyas a CONIT system bug .that 
seriously interfered with the experimehtal session. Two ,such situations 
arose: (1) inadequate handling of /'TIME OVERFLOW*' message from ORBIT for 
EU3 (Section D3.2) and (2) incorrect >handling of the BREAK request for EU4 
(Section 5) • In the secohd type of situation, a clear inadequacy in 
CONIT operation was ""perceived . This might be termed a kind of design bug 
as opposed to the implementation bug of the first situation. Again, vthere 



-:'.H- 



\ 



wore two such situations: (I) confusing message' when de 1 ay ^occuis 
• in, the host system response (see Section 1)1.2); and (2) Ihv Incorrect 
handling of a ^earch on terms found in a SHOW INDllX response (j^ee 
Section 1)4 .9) . 'a 

in the third type of situation, the supervisor simply decided to 
tcrmiiuite tht^ session before the !iU was fully prej)ared to do so on his own 

• This happened with HU2 when the supervisor advised him on how to get off- 
lino output' in order to finish the session. 

In other cases of CONTT, communications, or user problems and quest- 
ions, the supervisor said nothing or merely urged the user to attempt to 
Jumdle the problem himself. In no case did the supervisor i^nterfere with 
search strategy formulation or otherwise push the l:U to use a particular 
command or approach. . I 

In a working interf ace: environment , t'^e first tw^ types of situations 
would' be ironed out, after a suitable debugging periodX^ fact, all of these 
V problems uncovered in the ex}:»eriments were^soon corrected in CONIT. The 
problem with R02/'on the other hand, has been accepted as a partial fail- 
ure of the CON rr system used by the HlPs, although in fact EU2 was satis- ^ 
fied wi^i the results he had obtained online. ' J 

V 

/ • 3 » 4 . 3 !jsgL!Ail^-?igy7^^^ Coironands . 

In Evaluating the EU'^s mastery of CONIT commands, we may once 
more look at both negative and positive aspects. On'^the negatVe side, as 
is detailed in Appendix D, four of the EU's each made at least a few -mis- 
takes' in using CONIT commands. Oiv the positive side, each EU did, with 
the help' of the system, successfully use, -either at first or eventually, 
each basic command f' Furthermore,- with the exception of EU2, who blundered 

• into the use of several commands without really understanding them, each 
Eli ^ arrived at a reasonable understanding of all the basic commands, at 
least in their simple forms. , 
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It IS iiist ri!ct ive to' cons ii^lcr the mix of commands used by the l-.U's, 
Table 3 shows the eight most coimnonly used commands issued by the liU's. 
It may be noted that, while all lilies were in a learning mode, weJ 1 over half of 
the commands were "working" commands, i..e., commands requesting searches, 
search terms, or document output as opposed to conmiands simply asking for 
explanations of system use Or requesting continued waiting for a host res- 
ponse (YKS) . ^ ^ 

Table 3. Most Commonly Used Commands 

Conmiand ' Average Number of 
Ncime Usages, per User 



FIND . . 


15. 5 


YES (response to the wait question) 


12.8 


EXPLAIN 


10.3 


SHOW (Documents) 


9.2 


COMBINE 


'6,9 


SHOW INDEX 


3.8 


PICK 


2.8 


BREAK (key hit) 


2.7 



While all the basic commands were used by each there were a number of 
specialized commands and explanations which could have been more helpful if used 
by more of the EU's. The SHOW REVIEW command was used by only two EU's; 

others would have been able to keep track of . previous searches more easily 
if, they had used it. None of the EU's used SHOW TITLE, although that could 
have speeded up browsing through document output. EUl did not use SHOW INDEX 
or SHOW ALL two functions .that could have helped her search strategy for- 
multation. . ^ 

Three EU»s did not avail themselves of the opportunity to get offline 
output. The explanations' for developing search strategies beyond the 
initial E FIND \ received scattered usage. Table 4 shows the number of EU*s 
who requested these explanations. . 



Table 4. Use of Search lixpl aiiat ions 

HXPLANATION , NUMbFR 01- FA)'S RHQimSl ING 

R FIND 5 

0 FIND AirmOR 3 

, H FIND BGrrKR 3 

E Fir/i) MORE 1 

Jl gOMBINE 4 



As will be discussed below under search strategy analysis, the 
ef fectivenes s of use is related to the thoroughness with which the GU^s 
sought and read the explanatory information available online. 

-To delve fuirther into the effectiveness with which nu*s used CONIT, 
we may consider the time required to perform particular tasks. One approach 
to this analysis is to look at the time measurements in Table 2, and in 
particular, at time differences . In recording the "timc^ to first PICK", 
we are/measuring how long it took the EU to get, read, and digest the 
initial online explanations about system use and Commands particularly, 
EXPLAIN and PI)CK commands as well as getting sufficient information 
about databases in order to select one. The average for all six EU's was 
nine miijutes. The Group A EU's (with offline instruction) did considerably 
better (seven mini5bes) compared with. the Group B (no offline instruction) 
BU's (12 minutes). It is tempting to attribute this differende^ to the 
availability of database information to Group A prior to the online s^s- 
sion* However, a detailed review of the individual EU session, which we 

shall now give, casts some doubt on this simple interpretation. . ^ 

-I ■ 1- ■ . ^-j 

~ It does appear true that each of the EU's was' able to get and read 
the initial explanations with reasonable dispatch. Also, .none of the Group 
A EU's used any SljOl^ DATA command to get online information about data- 
bases, whereas the Grpup B EU's used an average of three such commands 
each. However, if we l^ook at the time to pick for the individual EU's, ^ . 
we see that it is only the time for EU5 (2J minutes) which is significantly 
& .' • . • ^iffetent frcwn that of tjie oth^r fiU'sp and is the cause for this large dif- 



ferenCe between the two groups. When the session for BUB is reviewed 



(see Appendix D) , it is seen that a major reason (or the \on^\ t imc- 
to-pick by tlie 1-U wa,s his tlocision to got additional information 
about one particular databa*se and the difficulty in g^-^^i'^i^ that' infor- 
mation because of telocommmrications problems and a mi sroad i nj> . of t,he 
database number. Hxcept for these problems, the lUJS t :i ikv- 1 o-^p i ck would 
have been about 10 minutes, and Group B' s average won J[d .have beeir close 
to Group A's seven -minute average. 

On the other hand, t\\c one liU who was most helped by t,he offline 
database infonnation was IUJ2. Unlike" most of the others, he had little 
a priori knowledge of any oiUine database, and he used the alphabetical/ 
index to databases by professional fields Appendix C) to learn that 

aeronautical engineering was covered by the COMFliNDI-X (I^NGlNHIiRlNG INDHX) 
database.. Yet it was who spent the second most time to pick (nine 
minutes ) of any of the KU's. The explanation for this anomaly is merely 
that HU2 spent more time on the early command explanations due tojus 
many typing errors and other confusions. 

An analysis of the utility of offline database instruction suggests 
that this method of presentation could save two to three minutes of on- 
line instruction for users like our ElPs. While the overal^ system cost 
might be somewhat lower with offline instruction, the total user*s time 
might be expected to be about the same, assuming similar information 
available from both hiedia. Additional comparison of offline and online 
modes is given below in Section 3.5.3. 

^ This analysis illustrates the need to look behind the siinple averages 
to the individual circumstances in order to interpret the statistics 
properly. In particular, considerations of individual background, needs , 
and errors, as well as system problems, n^d to be taken into account. 
This is especially true in the sit|aation where relatively few individuals 
are being grouped in each statistical group. 

The time from the first PICK .to the first I^IND 'command (2b - 2a) 
includes the time to make the connection to the database and g6t online 
instructions on searching, read the instructions, and prepare the first 



search. Vho. avoi^age t imo for the six liU's was 10 miiuitcs. !t might 
ho oxpot^^tod that (Jroup A, who had t htv pro- soss ion offline inst met .1 oi\ , 
would he faster. However, Group li is actually faster on average; seven 
minutes compared with II mi.iuites. Here again, wo must look behind the 
averages. The longer average for Group A cai^ l>e traced to l:U;^, wfm took 
27 miinites ii^ this per.itxi. When we analyze the reason for this length 
of time, we find tliat was the only I'D who requested all of the on-^ 

line search instnicti ons , despite having also soon the offline instructions 
f^J3 was meticulous in his utilization of both offline and online instruct- 
ions; in this respect, it may f)e noted that he also spent tlie most time 



(36 minutes compared to an average ,^of 26 minutes) in usil^g offline iastruct 



ion. But liU3 was by .far the most successful of the l-U's in developing 
\]\ effective search strategy a!ul in achieving results, "as demonstrated 
by the recall figures. It is therefore clear that we must measure accoinp^- 
lishmeiit, as well as time, to some particular juncture. 




Other time differentials of interest are the time between the first 
search and the fi.i'st document output average six minutes ~- and between 
the first document output and tlie first useful document found average 
10 minutes. While there is considerable variation among the individual 
EU's for these measures, the average^f igures give us at least some ind- 
ifcation of the time and effort required to develop the search in these 
respects. 

An objective measure of accomplishment might be related Yo.Upw busy 
the users are in terms of number of commands issued, or number of pages 
of typescript generated. When these figures are normalized with respect 
to time; we see that the KU's averaged 0.9 commands per minute, and IS 
pages per hour. There is some degree of variation of these figures over 
the different EU's. EU2 measured lowest in both categories and, in fact, 
can be $aid to have accomplished least in both understanding and retrieval 
results. Sigwfiifier, the next-lowest measures are for EU3, who did the best 
in terms of system understanding, effective use, and retrieval results. 
The highest figures were attained by FA)1; but, a^eview of her session 
indicates .that she may have missed some opportunities by moving too fast 
without taking tjufficient time for reflection. Therefore, while these 
measures of busyness may have some correlation with accomplishment, it 



is not always necessarily a positive corio la t ion . 

Before giving our final conclusions oi^ tu)w j;ooJ (ho iiUoifaco was 
in heJping users learn to use coimnands, and on wfvat iUc potential is 
for improvement^ Wo sliall look more carefully al lUo accompl isfnnont s ol' 
the liU's in teiMns of retrieval i^esults and searcl^ strat ej.> ios . - 

3.4,4 Search Strategy Consideration's 

The ajialysis of the retrieval cfl'cctivcneisL^ in termt? oi' 
recall for a given effort in time spent, compared liU results witli those 
of analyst searching. This analysis led to a inimber of tentative con- 
clusions: 

1) Users can get some relevant information solely through the 
interface techniques. ^ 

2) The natural-language, keyivord/stem approach to searching is 
one important element in making the interface techniques 
successful . 

3) ' Although there was a considerable amount of satisfaction by 

EVPs, the retrieval effectiveness in terms of recall per- 
centage was only moderate-to-low. 

. 4) , Improvement to interface ca^bi litres is possibfe through 
enhancements and extensions to existing techniques. 

In order to justify these positions , we shall look at individual sessions. 

The session of RUl proved to be instructive from the viewpoint of 
search strategy -..formulation and will be reviewed here in some detail. 
The topic for RUl was "the interface between social service agencies ai]d 
public libraries and how their information sources complement each 
other". EUl was considering this topic for a research' project, but she 
had not yet done any searching. She had, however, broiifsed for about 10 
minutes through the ERIC thesaurus of descriptors (ERIC, 1972) before 
coming to the experimental session, and had brought with her a paper on 
which she had written the following ERIC descriptors as potentially useful 
to her search: 

1. -lipraries ... < * 

k/- ■ • •J 

■ .3, .: social services ■ || . .- S^f"' " " ' Cl ' ■ ' 



^, cominuM.it y services 

human services v ' 

0. social afjencios 

7. public libraries ^ ' 

8. library reference services ^ 

9. library services 

10 . commufi i ty In format i on Serv ices 

11. information sources 

■ ■ ■ •■ • • • ▼ 

|:U1 Started out searching on liRIC (see Table S for list of search-related 

coimnands) using the keyword stem approach suggested by COMlT with tho 

search on the word stem ^Mibrai:". Unfortunately, us described above, 

the delay message at this point confused GUI and she did not wait for - 

search completion. Thinking that it was the stemmiiig that caused the delay 

t 

problem, EUl reverted to full-word searching (U3 \\\ l^ible 5). She then 
tried two~phrase searches (U4 and U5) . The first got null results because 
it is a mixture of two RRIC descTvlptors, but'not a descriptor itself; 
the s'econd was aborted because off;^he delay -messaget problem. ^-^ 

At this point, GUI went back to the keyword search' strategy which 
she employed for th^ remainder of the session. GUI then retrieved and 
looked at (U12 and U16) the standard document output for docun\ents from 
the. 5tfarches "libraries: AND social:" and "public: AND libraries: ANl) 
social:". No documents appear very relevant ^mitl"TA)l decided that she ' ' 
needed to bring in the "int^eraction" ^ohcept. One part of the problem 
is her mistake in using set numbers which prevented the results from 
being intersected with the Search on ."service :". 

Following through on her hunch, EUl searched on "the word "inter- 
action:" and intersected the result wit^fi the previous searches (UIQ) ; 
this left just two documents. On seeing the ,^ltle of one of these two,, 
"i^et^ormance Guidelines for Planning Community Resource Centers", she 
felt tmit she had verified her hunch . She. th€|n requested a. search on ^ 
"cooperation^' as a synonym for "int<^raction" and created (U22) SET 18, 
which is the combination of "cooperation: AND public" AND libraries: 
AND social:". On looking at the Stahdatd output for thjSi first five doc- 
ui)i6nt^» of the 30 doctiifnents in this set, EUl felt.thAt four of these 



Table 5. I-IJI Search Strategy 

Cmmil)_ > RHSULT 

Ul. PICK 56A --> Connected to F.RTC on 'DIALOG 

U2. FIND LIBRAR --> DHLAYF.D RHSPONSF. • , 

IJ3. FIND LIBRARIKS '--> SET 2 (7447 Documents) 

U4. FIND COMMUNITY SERVICR AGENCIES --> SET 3 (0 Docs) 

U5. FIND SOCIAL SERVICE^ AGENCIES ---> DEIJVYED REPONSE 

U6. ' FIND PUBLIC ---> DEUYED RESPONSE 

U7. FIND SOCIAL SET 6 (44,534 Docs) , 

i 

«U8. COMBINE SET 2 AND SET 6 ---> SET 7 (526 Docs) (LIBRARIES : AND SOCIAL:) 

. U9. FIND SERVICE — > SET 8 (37,572 Docs) 

UIO. gOMBINE SET 6 AND SET 7 SET 7 mistake, meant SET 6 and SET 8) 

*U11. COMBINE SET 2 AND SET 9 SET 10 (= .SET'7 continuation of above 

mistake) 

I Ul2, SHOW > Standard information for first five dociunents of SET 10 shown 

U13. FIND PUBLIC— -> SET 11 (41,738 Docs) 

U14. COMBINE SET 2 AND SET 11 > SET 12 (2,732 Docs) 

U15. ^ COMBINE SET 10, AND SET 12 ---> SET 13 (222 Docs) (PUBLIC : AND LIBRARIES : 

AND SOCIAL:) 

\)l6.. COMBINE SET 9 AND SET 12 — SET 14 (= SET 13) 

Ul7, SHOV/^--— -> Standard information :^or first five -document ||d|hown. 

U18. - FIND INTERACTION — -> SET 15 (9401 Docs) 

■ U19. COMBINE SET 14 AND 5ET 15 SET 16 (2 Docs) (INTERACTION: AND PUBLIC: 

' . - ^ ' AND LIBRARIES: AND SOCIAL:) ^ 

. U20. SHOW > ^ Standard information for the two documents shovm 

■ ' ■ ■ ^ '" if ' - ' 

, U21. FIND COOPEltATION % SET 17 (7155 DocS) ' 



U22. ■ COMB INI- S1.:T M AND Slil' 17 ---> SliT 18 {:W Pot s) (C(X)PI;RA T I ON ; AND 
PURLlC: AND LIRRARTHS: AND SOCFAl,:) 

U 23. SHOW > Standard information for first five documents shown 

U24. S S18 ALL Dl-30 OFF > Offline print for SliT J.8 (all inllormation) 

requested , 

U25. PICK 55B -- > Connected to LISA ORBIT 

U26. "find public SF.T I (4379 Docs) 

U27. FIND LIBRARIKS ---> ' SFT 2 (11.709 Docs) 

U28. COMBINR SBT 1 AND Sl:T --^ SET 3 (3667 Docs) 

U29. FIND CONff^NITY ----> SRT 4 C3^3 Docs) 

U30. FIND SERVICE > SOT 5 (3837 Docs) ^ 

. U31. COMBINE SET 4 AND SET 5, > 'SET 6 (145 Docs) 

U32, FIND COOPERATION! --- > SET 7 91178 Docs) 

.U33. COMBINE SET 3 AND. SET 6 -'--> SET 8 (85 Docs) " 

U34. COMBINE SET 8 AND SET 7 — -> , SET 9. (10 Docs) 

U35. SHOW >. Standard information for first five documents shown 

U36. S S9 ALL Dl-lO OFF > Offline print for SET 9 from LISA search 

J requested (all information on all It) documents) 



these (locuments arc oithor highly or mcKleratoly rolovaiU; tliis JucIkiikmH 
is based on seeinj^ titles such as ^'Cooperation Amojip, Unlike Institu- 
tions for Today's Learning Force'^ and "Comnuiaity Problems in I'ive West 
Cent ra 1 Count ies". % 

\ Believing from this sample test tliat she had found a sizeable 
number of rol evant'jocumont s , lilM request ed (1124) that all informatiori for 
all documents, be printed ofliine, ajid she Lhen proceeded t-o search 
another database: LimNFO (I/ISA) . (h\ this database, HUl directly and 
confidently executed a keyword searching strategy; this contrasted 
strongly with the fumbling development of such a strategy on the first 
database. One variation is the substitution of "commuirity" for. "social" 
as a search term; we did not determine a reason for this substitution, 
but the use of this word in the titles of apparently relevant * documents 
seems to be a factor. 'The set resulting from this strategy", "SHT 9, "has 
nine documents, and as Table 6 shows, lilll rated the first five "highly 
relevant" on the basis of the standard information 'from (U3r>) . She then 
requested an offline printout of all the information for all nine docu- 
ments. Peeling she had retrieved all the documents she needed for , this 
early stage of her research, and not wilting to "use up too much ^valu<ible 
computer time", ElJl then terminated the online session. 

When analyzing this session, we looked at the of f 1 ine qn^intouts 
that EUl had requested. In contrast with the standard information 
requested online, these printouts included abstracts:' On the basis of 
this analysis, we came to two hypotheses: ^ 

. 1) the documents retrieved were not actually as relevant as 

EUl thought; and " . ] J 

2) the two keyword stems "librar:" and "communit:" by themselves 
seemed to be good in a coordinated search strategy. 

Searches based on the strategy of (2) , with some variations for ana^^sis 
purposes, were mkde in ERIC and LISA,^ibs nine other databases. Full-re 
cord output of the resulting documents, or sampled subsets where the 



retrieved sets were too big, were presented to Elfl for relevance judgment. 
The searches and judgments are shown inTable 6. 
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IJKiliNI.) rOR TABU-. 6 

public: AND .libraries: AND social: AND cooperation:' 

public: AND libraries: AND community: AND service: AND cooperation 

coiimiunit: AN!) librar: - - 



D: (social: fiW (servic: OR agoiic:)) OR (human: AND servic:)) 



AND librar: AND NOT communit: 
[C] AND NOT servic: 
[C] AND NOT librar: 
coiimmnit: AND inform: 
librar: 



Num 



ul)er of documents retrieved by search 
NE ~ Numbet of dociunents evaluated by KU 

MOD!: = Mode of search and evaluation: U = user search; A = analyst search; 

N - online evaluation; F = offline evaluation 
RELpVANCE: Number of documents evaluated with high (H) , medium (M) , 

low (L) and no (N) relevance' - ^ 

PRECISION: P^j = H/NE; = (H+M)/NE - • 

TOTALS: Estimated total number of documents of high (H^,) Or medium (M.p) 
relevance based on (extrapolation from) those evaluated. 



\ 

rho rol^^vance ovaluations and otfior reaction by lilll confinnod 
the t^o hy])othescb. As shown in Table (>> i'Ul downrotod t ro Icvanco 
evaluations from the ratings made online on tlie basis of iho limited 
information seen at tfiat time. Ap]>arent ly, lUJl fiad siinj)ly assumed 
(or hoped) tfuit the {)resence of the searcfi words on the retrieved doc- 
uments would imply the desired, relat i onshi ps among these words. Wlien 

~ ^ 

this assumption was consistent with at least one int eri)ret<'it ion of the 
titles, I:U1 took tfiat as confirmation of the assumption. 

In fact, however, tfie "coo])eration'' mentioned in these documents 
was generally intra-1 i hrary cooperat ion, not cooperation between the 
libraries and social service agencies. What is more^ the words "social", 
"service", and "communit)^^" generally di(^ not refer to agencies other 
thaji libraries. Furthermore, there appears to be relatively little, if 
anything, accessible in the available databases specifically on library 
cooperation with social- service agencies. The best that can be done for 
this topic ^ and what EUl turned to, is to redefine the topic of interest 
as "the services offered by libraries which support social service agency 
efforts (whether or not there is any specific librai^y-agency interact idi) 

The analysis of this session supports several of the assertions made 
in deciding how to assist inexperienced users to search heterogeneous 
databases. In the first place, controlled-vocabulaiy searching is gen- 
erally more difficult and less effective than free-vocabulary searching, 
especially for the inexperienced user. Even EUl, a library- science stu- 
dent, had trouble trying to use the ERIC thesaurus. As a matter of fact, 
there are two ERIC thesaurus descriptors that, when intersected, give 
a moderately 'good (high-precision, low-recall) search strategy: public 
libraries and ccnnmunity information servi ces. One problem, especi^ly 
for^the inexperienced users, is the difficulty of finding the good terms 
and determining that they are good. Analysis sh6ws that intersection of 
these two search terms, while yielding a high precision (>0.9) causes 
recall to drop undeir 10 percent of it's value on the more optimum k*?yword 
search. There appears to be no way to bring the recall back up to the 

» St * ' 
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keyword search level without using many adclitioiia! descri.i)tor terms ' 
many more than the 11 found by liUl. Other terms noted from relevant 
documents, but that were found by FIJI keyword search A but that would 
be mi;>$ed by this simple descriptor search, include: 'Mnformation centers" 
"conununity health service's*^ and "Infonnflf ion sources'\ Of course, as 
more terms are included in the union (ORihg) of the social-services cqn- 
cept search, precisi|)n will decline. 

The control led- vocabulary type of search has even more severe prob- 
lems for ^t his topic, in trying to extend it to most other databases. 
LISA is one database that does use the descriptors "public libraries" 
and "community infonnation services" ; however , as in FJUC, the inter- 
^ection search on ^hese two terms seems to' yield a very low recall (<^.l) 
in USA of .the estimated 293 relevant documents. A brief analysis of ' 
Other descriptor terms used in LISA uncovered no descriptor search strat- 
egy, even a complicated one, that would yield anything close to moderate 
recall ( ^ .5) at moderate precision .2). 

The NTIS database descriptors are somewhat similar to those of KRTC. 
However, the reports in NTIS are essentially a subset of those in HRIC 
for this topic, and so searching this database in any mode does ,not aid 
retrieval once ERIC has been searched. I'he control led-vocabulary of 
ERIC carries overJPRorly, if at all, in any of the other eight databases 
searghed. As with the LISA database, no effective sul:tstitute controlled- 
vocabulary search could be found in any of these databases, but the l^ey- 
word strategy was reasonably effective for each of them. 

An optimum, or even very"^ffective, free- vocabulary keyword search 
strategy is not necessarily o^sy to develop. It took a fair amount of 
analyst time to determine tnat the search ."communit :" and "librar:" is 
close to >optimum. It takes test searches and document output, to deter- 
mine that words like "cooperation", "service" and "social" are more 
hurtful than helpful when us^d in addition to, or in place of, the ^ 
optimum search words. Also, it takes a redetermination by the end user 
that libraries other than those designated "public" (e»g., municipal, 
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county, college, ^etc , ) may provklo services tluit are relevant to tlio ^ 
topic and, l!\crcf'orc, t!ie limitation to "i)ubiic*' lihraivies is uiuieces- 
'sarily restrictive. While sucli analysis may be more or Joss necessary 
in different situations for either modes of searching, our analysis of 
searching for lilJl and the other HU^s ifidicates that tlie free- vocabulary , 
keyword/stem approach is easier, especially when- sparclung across 
muiiple databases. 

The figures sl)ow the need for multiple databases to achiev^ high 
recall. Us^ipg only one database reduces recall by 0.28 , llie^top ^wo 
databases miss eight percent of the recall base. Fully six databases 
are needed to avoid missing important numbers of documents- Gven small 
numbers can be important if they bring in a different perspective or 
document type. It may be noted, however, that the precision of the key- 
word search and the relevance of the relevant documents (ratio of IPs to 
M's) goes down in the peripheral databases. 

Using the offline instructional materials, KUl, who did have some 
prior knowledfe of the databases, was able to select most of the sig- 
nificant ones. She was also able to select the t?wo most useful databases 
in rank order for th^ online session. Two databases she selected, ^ 
Psychological Abstracts and Public Affairs Information Service (PAIS), had 
negligible information on her topic. She failed to select three general' 
databases with moderate amounts of information: LIBCON, SSIE (Smithsonian ^ 
Science Information Exchange) and GDI (Comprehensive Dissertations 'Index) . 

Several searches (see Table 6, searches D, E, F, G and H) were run 
to determine if the actual recall base was much larger than the one deter- 
mined by the basic two-word stemmed search. Results of these searches sug 
gest that the actual recall base is probably within 20 percent of that 
found with all of the analyst searches. Search F shows that a concept 
subsumed by a database (here ''library'' in LISA) may be left out of the 
search statement in which case recall is increased by five percent at 
the expense of a 0.08 percent drof) in precision. 
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ScveiMl additional points oi\ the search, strategy can be mentioned. 
In the first place, tmneation seafciiing on word stems e!(Nirly aids 
retrieval; "librar:" picks up ^'library" and ibrar ies*'; 'k^ommun i t : 
retrieves '^community" and ''Coinnuinities'*; '^serv.ic:*' covers ^^service'*, 
"services", and "servicing", etc. Secondly, there are possibilities i'or 
special search techniques for improving precision; for example, exclud-^ 
ing documents on "community colleges libraries" can raise precision 
significantly in some of the databases without reducing recall noticeably. 
Increasing the coordination level does increase precision significantly, 
but at a major cost in recall (witness searches A and B) . 

The detailed review of the analysis for I:U1 given above exemplifies 
tHe 4:ind of search strategy analysis accomplished for the other l-.U's. 
We shall now ^ive highlights of search^ strategy considerations for the 
other KU's. 

EU2 ^ . ^ ' ■ ' 

i . 

In view of the tentative ancT exploratory nature of^ the topic as ^ 
perceived by HU2, we did -not perform an extensive se£^ch' strategy afnaly- 
si^, EU2 \\\t upon the quite reasonable search term "wings:" for his\^' 
topic , ;Ving-section designV, In thfe Engineering Index database, clearly 
the best one for this topic, this search gives a precision ^of. 0.25, and 
presumably^N^ very high recall. 

Coordinating the search "design:" with "wi n ig o ^f '^ raises precision 
to approximately 0.6 at the cost of-halving the recall. Contrarily, it 
is interesting to note that a truncated . search on the stem "wing:" raises 
recall by about 80 percent, with relatively minor losses in precision • 
Similarly, truncated s^SSbches add significantly to exact-match search- 



ing; for example, "des:^gn:" recalls 30 pep^i^ more than "design". 

A control led-vocabulary searchiiYg strategy is possible here using 
two terms from the Subject Headings in Engineering (SHE, 1972) thesaurus: 
"Wings and Airfoils" and ^^Design". \3earching on these terms instead 
of the individual wdrds in the basic index appears to raise precision 
somewhat, at the expense of a serious degradation in recall; for example, 
the search on "design" as a descriptor recalls only 28 percenj. as much- 



as tf\e search on the word 'Mesigir' in the hasic iiulox. 

It also^ appears cons ideral)! y simpler Tor a user^to select words 
from' the patural language than to have to look up th^ appropriate head- 
ings In a thesaurus, especia 1 ly ' (as is the case here) where the thesaur- 
us IS not available onl ine. Note that th i s^keyword/bas.ic-;i ndcx approach 

,aV*sp avoids forcing on the user considerations such as thp distinctions 

. , \ - . . ^ ' " I 

between main headings pnd subheadings, controlled terms (descriptors), 

and'^free terms ' (identifiers) , dQ;ijCJ\iptor phrases and descriptor words, 

and title and abstract woixls . We may a^so note that these distinctions 

are further confounded by a few tyi:>ographicalf errors and/or spelling 

V . .; 

variations th^ft iiave gotten into the database index fW the descriptors; 
some examples are ''win^ga. and ' airfoi Is", "wings a^airfoils", "winj^s^^^uid 
aerofoils", and "wings and air foil". While there are only one or two 
documents involved ih each variation, each such variation appears as a 
separate entry in the online index and, collectively, they tend to 
clutter up the index and make the selection of' search terms more obscure. 

EU5 ^ 

J 

The two databases searched by EU3,,Vi'S^CI (Social Sciences Citation 
Index) and SSIE offer an interesting contrast in indexing. SSCI has 
very shallow ijindexin^ (title words only), whereas SSIF, has very deep 
indexing (up to 200 words or more per document), including an elaborate, 
hierarchical, control led- vocabulary thesaurus of subject terms. 

EU3 carefully read both offline and online instruction on search 
strategy formulation and moved directly to what turns out to be an 
effective, sifnple keyword/stem search for this topic: "world: AtJD model:" 
This search give^ 112 documents, of which 70 are rated H and 13 M, for 
« precision of 0.74. Substituting "dyna^nic:" for "model:", EU3 found 
25 additional relevant' documents out of 37 retrieved. However, the use 
of "international:" for "world:" and "simulat:" (e.'g., simulation) for 
"model:" in three other searches found only three more relevant documents 
EU3 felt that he had fairly well exhausted the possibilities of effective 



subject searching with the above searches. hater analyst searching hoj:c 
this out* Although citation searching fiiuls additional est iniat ed ' ^KK) 
relevant documents, there f»f>]:)oars to he no sample subject searching 
strategy that would retrieve many of these with reasonable precision. 
Citation searching, which was not availabl(^o the lilJ's in the virtual 
moile, waS rather effective for this topic, which has a few kqy authors 
such as J, W, Forrester, f). M. Meadows, ami C. W. Churchman. Citation 
searches on these three authors yield 1301 documents with a precision of 
0.3, it is estimated that the actual recall base for SSCI might be as much 
as twice as large as that calculated from the searches done. ^ While it 
way Id be quite difficult to retrieve many of these 'Miidden documents" 
with subject searching, we suspect that a sizable number could be retrieved 
with citation searches on other authors. 

The search "world: AND model:" in SSTK retrieves.'606 documents with 
low precision (estimated 7%), Coordina^t ing with the word stems "social:" 
or "econom:^ raises precision to the \).2 to 0.3 range with very little loss 
in recall. This^illustrates the possible need for more highly coordinated 
search strategies in databases that are deeply indexed, if the simpler ^ 
Strategy retrieves too many irrelevant documents. ' 

EU3 was seek:\ng to achieve as high recall as possible and he wanted 
to see even those documents he had rated L (low relevance). In this situa- 
tion, at least 14 databases are seen to be important. Based on the offline" 
instructional material, EU3 had selected seven of the more important ones 
(SSCI, SSIE, GDI, ENERGY, ENVIRONMENT, LIBGON, and 'INSPEC-EE) although. he 
had time online to search only the two most important ones. EU3 selected 
only one database (GRANTS)- that later proved relatively fruitless. Analysis 
uncovered seven other databases, of moderate utility (estimated as having 
more than 10 documents of some relevance): NTIS, PAIS, ERIG, BUSINESS (INFORM), 
SOCIOLOGIGAL ABSTRACTS ,■ MANAGEMENT , and OIL (TULSA). Five other databases 
were found to have some fewer numbers of relevant documents. 

' We note that for this topic searching on .controlled -vocabulary terms 
as such, for those, databases where they do exist, seems generally fruitless. 
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. It is also of intorost to note that was hampered in l\>llowiMf> CONIT 

instructions to find appropriate s^arch^terms in SSII- because the 'MMUNT 
RJM/* coiiunand into which CiONTI' regularly translated the "SHOW ALL" request 
does not output index terms in this one ORBIT database. This problem, now 
fixed, is another example of how handling individual database peculiarities 
can be important. 

[Xie to various CON IT bugs and a s>^steiii crash (see Appendix D) , UU4 
retrieved only one reTevant document on his topic. It appears that LU4 
would have achieved good success with his attempt at following the keyword 
approach if system problems had not intervened. Keyword-stem searching 
appears to be a highly efficient technique for avoiding the complexities 
of the multiple and overlapping controlled- and free-vocabulary indexing^ t 
used in the prime database for this search rNSPKC-PHYSICS . For example, 
if HU4 had been able to redo — as he had started to do his search: 
"helium: ANCfion" AND electron excitation:" as "helium: AND ion: AND 
Vlectron: AND excitation:", he would have rep:^eved'28 relevant docu- 
ments instead 'rtf one, although precision would have dropped front 0.33 
to 0.04, ^ , . 

Precision could be enhanced without much loss of recalj, especially 
at high jrelevance, by fOrther coordinating with the concept "cross section"- 

What separated the m6re highly relevant documents from those les's' relevant 

i 

were often such intangible or hard-to-search concepts as (1) a more comp- 
rehensive analysis, or (2) energy levels (e.g., relativistic effects not 
wanted) . ' / -t> . 

EU4 ^eventually was able to follow onlixie instructions with effective- 
ness, although he never did use SHOW ALI/, which could have helped expose 
indexing usages. Online instruction easily led EU4 to s^ect the PHYSICS ^ 
database, which was his main source in jjpinted form for regular library 
searching. He also used the SCIENCE-CITATION INDEX (SCI) database online, 
because he knew the citation feature could h(?lp with "forward (in time) 
chaining";^ however, he never did get to use that feature. Based on his 
library experience, he thought the CHEMISTRY database was "too spotty" to 

■ < 52 ■ • ■ ' 
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bo wcl^n soarcluiig. However, his later ova 1 i!;it, i (>h*% on analyst soaichos 
did iiKlicato the luimber of nu>d(>tK!t.oly releVi-ml document cohid be ^/ ' 
doubled by searching tfils database, I.UJ4 fuKl sojne interest in bij^,h ixu.alt 
for his tKo.s i SL.. bjbl iography , bujt. he was more interested in trying to/ 
insure tfiat he had not.Mnissed any very ixvlevant documeiits. 

IU)r> did moderately well in developing and executing a searcf) strat- 
egy but he was hampei*ed by three kinds of problems. The fii'st kind inVblVOd 
system bugs and tfve secqnd kitul rel^ated to his difiiculty in following - 

certain CONIT instructions . \/Botf) kjnds of problems -~ see Appendix 0 for ♦ 

/ ' ■ ■ 

details — had the effect of slowiT)^g liU!^ down somewfiat and irritating frim^ 
but were not crucial, in tfiemselyes, in hir^d^ring scarcfi strategy develop- . 
men t . ^* 

^ ■ ; ■ . ■ 

The third kind of problem, whicfi had two aspects, was in the area 
of search- strategy development itself. fhe first aspect of this problem 
was his continued attempts at full-phrase searching (e.g., "hearing loss", 
"abnoiTnal hearing", and "interaural time".) Altfioug^h none of these was 
successful ("hearing loss" would have worked in Mi;I)l/INK, but was tried only 
in SCI), and his keyword searching was at least mqderfjtely successful, he 
seemed unconvinced that phrase searching was a poor strategy. Analysis of 
this session suggests some reasons for this reluctance to stay with key- 
wording: - - 

1) Subject phrases are "natural" to most users, based on their 

, experience with manual systems. ^ ' . 

2) The SHOW INDEX response showed that phrases were used (EU5 would 
have been more successful if he chose only such phrases and not 
simply invented ones that seemed reasoqabi^Ki .) 

3) The CONIT bug in selecting index term tags for searching (see 
description Onder EU4 session ifi Appendix D) cropped up when EU5 
tried to select a tag for a one-word term; this failure may have 
inhibited him from single-word searching, to some extent. 

4) EU5 did not av^l himsel/ of some explanations (especially, 

, E FIND MORE) that would /have einphasized single-word searching. 

/ . \. 
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IJadoubt ed ly , (hoso cons iiloiat i ons aio not onitjuo (o this !UI; in f';u:t . 
tl^cy wore obsorvod in other oxpor imont n 1 sn^^^r.ions. 

The second a! i nVi CM 1 1 y in sea rch - s t t^m t egy I Oniuil at ion derives (rom 
i;US'Vl"^.i lure to use onouj.;h synonymous terms in searchiu}.; or to collect tliem 
in 'V^oncept "^und les'' with t lie aj)propriate Boolean connectives. Tierce 
component concepts caji be derived fc^r tins toj)ic: 

1) liear inj^ 

2) the binaural aspect of hearinj^ . ^ 

3) \ mpai rmen t t o hear i nj> . ^ 

y f 

Terms 'synonymous or otiierwise jxlated to each of these concepts, and . * 
later used in anaMyst searches, are: 

1) heai:i.ng ; hear; sound ; bi naura I ; deaf; 1 isten ; i^ea r ; aural ; 
in t or aura 1 ; d i chot i C ; 
- 2) binaural; dic^botic; lateral; locat:, local iz:, loca lis : /mask : , . 
dnter aural; ' . 

3) impair:, deaf:, defect:, aid:, loss. ^ \' 

1:U5 used about half of these terms, but because of failure to search by 
keywords only, and to group syuonyins by the Bool><|^n.OR (union operator) \ y 
before ANDING (intei'sect ing) , (^ily a v.ery few of the 200 combination ti'iples 
were searched. Analysis showed that most of these terms, and many of the 
combinations, were needed to avoid serious recall and/or precision failures. 
Some precision problems were nb^d in using truncated forms (e.g., "heart'^' 
matches "hear:" aitd "earth" matches "ear:") but fals^e drops tend to be 
excluded on coordination. . . , 

EU5 had a bifurcated criterion for determining relevance; On the ope 
hand, he was most interested in those documents where the' three concepts 
^jwere explicitly treated; on the other hand, he also declared relevant those 
documents with a good treatment of the binaural hearing aspect alone if he 
felt they could be helpful in analyzing the hearing- impairment aspect. 
Approximately 10 percent of the estimated 1200 documents in the recall base . 
were in the former, more restricted, category. Even so, these 120 represent 
a greater-than ten-fold increase in number over the 11 documents EU5 had 
in a bibliography he had acquired before the computer searching. 

• • \ ' ' ■ ■ xJ 



Only a few of the 31 relevant docuiiicnts thi\( I'.ns^ rouiu! on 1 i ne fall 
into t\\c first category. Many others from both calegor i cs wore incJudcvl 
in tlie searches that l;US did do, and he could have raised his recall Irom 
0.03 to about 0.1 If he had simply dumped some of these searches for of i- 
line output. That lie did not do so may be attributed to several causes. 

1) frustration from the vaT:ious problems encountered online; 

2) not wanting to extend the online session ijnduly; 

' 3) realization that the precision for the category-1 type relevance 
was rather low; and 
4) knowledge that the experimental supervisor/analyst might get ^his 
V output for him anyway. 

1:U5 vVas able to select five,higlily relevant databases from onliTie informa- 
tion: SCI, MEDLINl:^ LLBA (Lc^nguage and Language Behavior Abstracts), 
Psychological Abstracts, and 0131; he searched tlie first two of these online. 
Other databases yielding (more than 40) relevant documents were: Physics 
Abstracts, SSIE, and NTIS. 

In order to achieve liigh recall in tliese databases, it is necessary 
to leave out the impairment concept. (Actually, since LLBA implies a • 
language/Ii earing concept, only the terms related to the binaural concept" 
were used to achieve high recall in that database): With tliat strategy, 
precision ran approximately 0.16 to 0.4. When the th^^d concept is 
/coordinated, precision is raised approximately 10 to 50 percent (actually, 
by higher percentages if just category 1 relevance is measured), while 
recall is cut by factors ranging from two to 20 in most of the^databases . 
:ln sell where the indexing is very shallow, i^ecall^ is cut from 160 to three. 

EU6 

EU6 presents another instance in which a bifurcated (specific-general) 
topic became evident. The specific topic is. ''oral examinations as a test- 
ing technique in medical education". The more general topic is "curriculum 
design .philosophy irt medical education". The relevance and recall figures 
iti Table 2 reflect a combination of these two topics . The main problem for 
J:U6 was a failure to keyword, due to an over-emphasis on controlled vocab- 
ulary. » EU6 got two documents when his search "oral exciminations" "^in ERIC 



matched that tVoe-vocabu la ry ( ideii ( i T i c i) phi\i!n". *!'h i s may bo corisid(MnHl 
cither liiOky {\\\ that ho got: any hits at all oii a froo phrase) or unlucky . 
(in that a mill I'osult jnight have pu.slu\l hiiii into nood^d koywortl i nj;) - ••l'<> 
noted in a post-sossion ii^lerv icv^ that ho s])co i Ti cal ly l osist od CONl T's sug- 
gestions to do kcywording because he was "afYaid that seaixhes 'oii tei-ms 
like 'oral' aful ' exajii i iKit i ons ' would be too Ivroad.'^ lie went on, instead^ 
to try to use alternate control lod-vocabuiary (hRlC t:hosaurus) terms, 
with which he wa s prev i ons 1 y I'aiuiJiar. This >ippr(Kich is a dead end iov 
that topic. Ills search "oral communicat ion AND evaluat ioff luotluxls", for 
exajn])le, yielded crocnmants concerned with the evaluat ion of oral commmrica- ^ 
tion (other than for examination purposes). 

A simple keyword/stem expansion of HUG's initial phrase (oral: AND 
exam:) yields 6^19 documents at about (a j^oor) 0.03 pi^ecision. Also, the 
stem "test:" must be added as a synonym for "exam" or about recall is 

lost. In order to bring up the precision acceptably, one mUst cocw:*dinate 
with a set of OR'ed terms signifying medical or other higher education. 
The stem "medic:" is better (about 20 p^ixenf higher recall) than the 
Tliesaurus term "medical education"; however, about 40 percent will be lost 
if one does not OR in cin additional series of torms^such a^"higher educ^^- 
tion", "colleg:". "professional education",^ "business education", "physician", 
etc. It should be noted, that liU6 himself got a start on such a precision- 
e^ihancing technique. Also, precision is aided by NOTing "language" to av^id 
documents on oral language exams. ' ' 

Actually, a , much simpler and mt>re efficient strategy would be to insist, 
on an adjacency matoh between the stems "aral:" and either "exam:" or ".test:". 
This raises precision to over 20% by itself (NOTing "language" would prob-. 
ably raise it to over 50%) without losing more than a few relevant documents. 
HoWever, the CONIT language does not yet offer that option and, in any case, 
it is npt executable as such in the MRDLINF: or ORBIT systems* 

N » 

The optimized search-strategy . fomuilation in MEDLINE presents an 
interesting contrast. Startin^g with the same basic keyword/stem search, 
one needs for precision to bring in some aspect of education (e.g., search 
term "educat:" --medical or higher education need no^ be specified, sihce 
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the database implied inodtciiio) aiui/or ncj>i!to dentist ly (stem" "dent:") 
the chief irrelevancy generator in MliDLlNi: in place of "huiguage". in !{K!t:, 

It is possible to achieve some success with a cent rol 1 ed- vocnbu 1 ary 
approach 4:o the broader topic on curricujum design, as 1:116 attempted, but 
a keyword approacli is st.il J much bettv^jr. The stem "curricu l:" is much 
simpler and just as effective as OR'ing the approximately 3S TJUC Thesaurus 
tef^K^ containing that stem an leas t f^BVcral of wlii ch are required for 
good recall. 1U)6 cut his recaM close to ^ero by insisting on the term 
"educational philosophy", lie wanted documents that discuss the |:4Hlosophy 
of curricula!!! design, but this term is Just not widely applied in l-IMC 
indexing. -.^ v 

Acceptable levels of precision are obtained by coordiiuitiug "curricul: 
wit(i "medical^ professional, OR health ed." for 13RIC and with just the stem 
"educat:" for MEDLINE* (Note: "educational philosophy" a^ an additional 
coordination term actually reduces pi^ecision at the moderate and lower 
levels 6f relevance, while reducing recall by a factor of 100 or more.) 
1'here may still be more documents in these sets that EU6 wants. Extracting 
the more relevant ones would involve further precision devices: i.e., title- 
word seax^ching. 

^ Additional CONIT f.eatures that might have helped EU6 are: . 

a) E FIND MORE 

b) SHOW OFFLINE (at least to dum|)^>on)e longer lists for search- 
^ . strategy review). 

The databases selected by EU6 using CONIT explanations are clearly the best 
ones, in the order selected (ERIC, then MEDLINE). 
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3.S r.valnation of Kosiilts 

3 . S . 1 Gene ra ! 01f> sorva t i on s . 

Because of the .1 iniitod number of users and liliiited amount of 

use of the expor:iment^^l interface, our conclusiotis must be cons jdorod st ^1 

) 

tentative, at least with resj:)ect to some of the details, l-urtliennore, our 
3 nvest i gations have shown that the superficial presentation of quantitative 
results may be mis leaching , especially when comparing results from limited 
usage. Howevei', through an in-depth analysis of the individual experimental 
usages, which has been summarized above, we have arrived at certain prelim- 
inary conclusions which seem justified by tfie facts uncovered in our invest i- 
gations up to the present time, ^ 

The central fact is that the virtual-system/transl ating-computer-inter- 
face appears to ^enable inexi)er^nced .en\l users to extract infoniiation they 
need from multiple heterogeneous databases and systems without t^pcourse to 
human intermediaries, , All six CU^s were able to find some useful information^ 
within a reasonable^ period of time; the aV^rage time to get to the first 
relevant dociunent refercjice was 35 minutes. In order to assess properly the 
potential utility of the interface coilcept, and the various techniques employed 
in its implementation, we shall now analyzt^ the experimental results in 
greater detail . 

The basic design and instructional teclmiques in the expOi:i^ 

ii\terface proved sufficient to enable the users to learn the mechanics of 

CONrr use fairly quickly. Each EU used each basic command at least once 

during the course of her or his online session. While there Were a number 

of ihistajkes made in'uping the commands, and several of the more specialized 

commands and explanations received only limited uSe, we can say that five 

of tl)e s^x EU*s appearedy^o develop a mastery of the basic commands by the 

,^ " . ' . 

end of the session at the terminal. This result tends to support our Initial 

hypothesis, that a modular, structured command-language approach to ulser con- 

trol over moderately complicated interactive systems is d viajjie approach, 

provided that suitable attenjicm has been given to design /or simplicj^ty . 

of use and to adequate online instruction. Getting inexperienced end ^users 

online, to interact successfully with several exij^ting bibliographic retrieval 
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systems as wo have now tloinoiist ( ed , is an iinportaiil achicvoinont (hat 
some observers of llio iiiformation-retr i ova 1 sco^o had boHuii to douht was 
poss ib lo. 

Besides gotl iiig users onl ine and usinj^^ cominaiids in a p!'0])e!' mannoi , 
the interface system must be judged on how effective the seai'ching Iuimis out 
to be. Our analysis shows that this is a comj) 1 i catcd question, and many 
factors need to bo considered before a proper assessment cai> be made. The 
prime facts are that estimated fractional i^ecal l ranged from 0.01 to 0.2:^ ' : 
and absolute lecall from one to .171 useful document references for online^ 
sessions the duration of wfrich ran from SO 'miiuites to 12*1 minutes. As 
mentioned above, the average time to retrieve the first useful document 
reference was 35 minutes wltli a range of from IS to 41 minutes for the 
different 'hU's. 

The problem in assessing t^se facts is tliat there is no well estab- 
'HHislied standard against \4iich to evaluate them. On the one hand, we may say 
that enabling inexperienced end users, without lumian intermediaries, t-o 
access iiwy amount of information from the given retrvieval systems in the 
relatively short pcriotis of tinie experienced represents another majoi^ 
accomplishment. Furthermore, there is likely to be a significant class 

of potential users of bibliographic retrieval systems who find human informa- 
tion specialists either too inaccessible or too awkward to work with, and 
who have sufficient need and financiaT resources to be willing to pay a 
surcharge if required (see Section 4 for discussion of interface costs), so 
that the interface approach to access would prove a desirable alternative 
to access^ via human intermediaries. 

• On the other hand, the values for recall appear sufficiently Jow, 
and the duration of sessions sufficiently Idng compared to times that have 
been reported for information specialist searches [see, e.g., Ross 
(1979) and Elchesen (1978)] that one might call into question the relative 
effectiveness of searching via the kinds of interface techniques we have 
been investigating. In what follows, w.e argue that this kind of objection 
is premature and, probably, unwarranted at least for the potential of 
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the iiUerTaco ai>]>r(>ac!K 1-irst, wo note that many u^um's Jo not want 
or !UH)(I h\}ih recall or, in any rase, iwc suti^.ficd with moderate' or 
low rocnll'as loi^; a!^ thc^y [\cl a Tow I'olovant cUnMniuMits to fill out a 
bib I i 0}>rap!iy oi' to start or coiUinuo a chai!\ of research. ^^'Vhc six lill's 
ran (he^gaimil oT recall desires. i;i)2 an'd hUS were looking foi -hijMi 
recall; 1:112 wai\ted only a document or- two^ and the other liU's were 
scattered in the range between these ends ol^ the spectrum. In terms 
t.oT satisfying their immediate wants and needs, I'Ul, 1:112 and \A\^ did 
(juite well, 1;US and 1;U6 did fairly well and !;lh! did poorly, due to a fore- 



f shortened session caused by the MlH/ru;S compute!' crash. 

Although recall may not be the top ])riority for all users, it is 
still a very imj)0!'tant parameter relating to system ])cr f-ormance . The 
ability to achieve high recall may, at times, be traded off to achieve 
easier and/or faster response (see, for instance^ Marcus, 1978). There- 
fore we should f>/i'ther consider wheth^ recall is adequate in the intei^- 
face system. 

We should note that the recall estimates shown in Table 2 are, 
in fact, upper bounds in that the analyst searches, on which they arc 
based, are not fully conipreliensive . We have estimated that the actual 
recall figures may be 20 pei:cenf to 50 percent lower, depending on 
the topic involved. On the other hand, it is not clear how well even 
an expert information specialist serving as an intermediary: would do 
in terms of recall on these topics. Certainly, the analyst searcHies 
were at least moderately compl^icated , and required a fair amount of 
analysis, on average, to develop. In any case, additional experiments 
wit!) expert searchers need to be carried out in oi^der to resolve this 
question adequately. 

,"^.5.2 The Importance of S ear c h Strategy 

One clear conclusion from th6se experiments and certainly 
one that is not surpris^ing to^ anyone knowledgeable in information science 
is the importance of search strategy formulation. Formulating an appro- 
priate searcfi strategy is obviously an essential for successful search- 
ing in general and it appears to be a particularly critical factor in 
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ihc success which iuox|)or i eiicod users can achieve in iiit eiac t i nj> 
wit!) exist iun online dnta!)ases. It seems clear from tliese experiments 
that^ the adoption ajul execution of a better soaicli sti^atejiiy is the 
maili element needed to improve searcli erfect i veness as measured by 

recad .1 as a funct ion of t ime f rom a range of poor- to- fa i iv! y f^crbd 
to one of goo<l- to -excel lent . • y 

/ 

A companion conclusion -~ not nearly so obvious a priori, and 
still nqeding additional experimental and analytic investigation ~- is 
that a natural-language-based keyword/stem approach "to searcl\ strategy 
formulation appears to be the best approach for achieving moderate-to- 
higlt recall, especially for inexperienced users. In our experiments, 
this approach was consistently superior tQ one based on selection of 
cotitrolled-vocabulary (tllcsaurus) terms^xcept in Isolated cdscs where 
the latter approach may allow for a quick search achieving higher [pre- 
cision (usually at the expense of lower recall) than a keyword search. 
In at least some cases, the strict thesaurus approach is simply impossible. 

The advantages of the keyword/stem approach are especially signifi- 
cant for the class of user we are j^articulai^ly trying to satisfy: 
the inexperienced user who may need acce^l; to several databases and - ^ 
systems. For such users, it is especially convenient not to. have to 
struggle with unfamiliar control led vocabularies . Users can start with 
a natural-language expression ' of their topic the word ''natural*' is 
key and apply the sajne basic strategy across several heterogeneous 
databases and systems, with assistance from GONITIS common command 
language. 

It should be noted that the success evidenced by the experimental 

users with tl^is approach depended, in part, on two. related facts: (l) 
« 

CONIT provides certain automated aids to searching; and (2) existing 
online databases now are, for the most^ pilrt, "keyword indexed" — i.e., 
documents are posted under individual words taken from titles, aJ^Stracts 
(where available), and Subject heading phrases, as well as the multi- 
word phrases themselves. The CONIT automatic search aids include trunca- 
tion searching and searching upder all available subject, indexbs as 
the default mode of searchipg. Keyword indexing permits a natural- 
language approach to finding documents indexed by a controlled vocabulary'^ 
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as lonp as tliero is a roaso!ial)le overlap in words from these two 

f'onns of expressions (or just in word stems, where truncation searcli- 

\ 

ing is employed) , 

A large measure of the success of the experimental users may be 
attributed to the emphasis by CON IT on the natura 1~ 1 angu}^?ige keyAvord/ 
stem approach. The main limitation to even higher search success by 
the users was the inability of CONIT to encourage the users to adopt 
this apln^oach more completely in their search-strategy formiWation. 

3.5. 3 Instructional Medi a and Learning Modes 

There does not appear to be ^a clear winner in the contest 
between online and offline instruction ^'or database selection and search- 
strategy formulation. Rather, it appears that both forms of instruction 
should be included in an optimal system configuration. More important 
than the medium is the content and quality of the instruction. Users 
generally prefer online instruction, but a sizable fraction also want 
offline insti'uction, as in the form of a reference manual which, many 
feel, gives a more customary, quicker, and easier mode for referring ~ 
back to previous instructions than, for example, browsing back through 
lengthy and bulky computer typescripts.' In addition, some users prefer 
to begin learning how to use a computer sysTtem by reading a manual before 
getting online, where time pressure is more keenly felt. 

' The experiment^il results do not confirm either mode as being def- 
initely superior. Individual variations in users, their problems, and 
external factors, such as system problems, clearly outweigh the (often 
small) differences in effectiveness measures for the two groups of users. 

It was found that the information available online concerning data- 
base selec!l|:i6n was generally sufficient, at least for se/ecting the most 
important databases on a given topic.- On the other hand, offline informa- 
tion in a printed format could and did, on one occasion -- prove use- 
ful, especially for users not at all knowledgeable in the databases, and 
where the index to databases by professional field or topic identifies 
a database as covering a topic not obvious from the simple description 
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of the database for examj)lo, library ami iiironnat io?i sciciuc bciny^ 
covercHi by the r.RIC (education) database. The expected t^adeorfs 
between of fl ine and online forms apply here; e.^., offline can be less 
expensive in that* cominit or time is not needed to access the information 
whoreafk ofiline can be- easier to update, especially for disi)arate, remote 
terminal locations . 

Users of both forms tended to underplay some of the ^;enera! dara- 
bases such as IJBCON, NTIS, etc. It was shown that ail topics requir 
use of multiple databases to achieve high recall, althoupji there usually 
were one or two databases that covered a large fraction of the recall 
base. For some topics, up to 10 or more databases seemed necessary to 
avoid significant recall gaps. Tliese results underscore the need for 
access to multiple systems and databases as ])rovided for by CONl'f. 

In regard to instruction for search-strategy formulation, again we 
find no clearcut evidence that offline insti^uction improves retrieval 
results. We do know that users say that such offline instruction is 
helpful and they desire to have it available in addition' to online 
insti'uction . The development of good search strategy appears more 
related to how assiduously and carefully the user reads and follows 
the instruction, whether presented offline or online. The critical 
question, rather than the medium of the instirict ion, may be how much 
and what kind of instniction should be presented the user before he 
attempts his first search or is connected to the first database or 
even before he starts learning system commands. 

Generally, it appears that the three EU's who used the offline 
instruction did better iYi developiiig the keyword search approach than 
did the three who did not have such instruction available. In^light 
of our conclusion that developing the keyword approach is crucial to 
obtaining better search resuj,ts, it is likely assiuning the above 
appearances are borne out by further evidence that additional 
instruction such as that given to Group A Ell's offline would be desir^ 
able in the stages before users start submitting search requests. This 
possibility leads to further questions C9ncerning ,whether it might not 

i 



bo bet tea' iT a useV'^s inter'act ion with the systems shouh! be bi'oken 
up into several parts: o,j;., (1) a pra(^ice or learniiij^ period 



whore, for example, illustrative, or "cannecP', searches could bo tried out 
by the user; (2) an initial search |)eriod durinj^ which the real |)rob!em 
might be first searclied; aiul (3) a secondary search |:)eriod in wh i cJa the 
initial searching might be revised after reviewing results - iiicludiiig, 
perhaps, offline output from the initial seaixh. 
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4. PKOSPI'CTS von COMinnHR INTIiUl'ACliS 

As explained below, we conclude that there are excelleiU prospects 
for both the immediate and long-range terms for computer iiUerraces to 
enhance the cost-effective utilization of interactive retrieval systems 
while satisfying end users* inforiiiat ion requirements. However, as showi^ 
in the preceding section, improvements to the experimental interface are 
desirable in an operational implementation of the v i rtual -system/t rans 1 at i ii^ 
computer- interface concept. Possibilities for some of these improvements, 
as well as other considerations related to interface costs and benefits, 
are discussed in this section. 

4.1 Iiiterf »c^__i|llPJ2.Y.^I55r'^A 

4 . J . 1 Re liability 

One obvious area f orximprovement is in reliability of opera- 
tion, especially in the context of hardware and software subsystems that 
are not- always reliable themselves. The tonn "robustness" has been Used 
to characterize systems that meet this criterion of reliability in the 
face of adversity. In our experiments, the problems to users of unreliable 
system elements were manifest. The difficul ties -engendered by such unre- 
liability seem especially pernicious for users in the learning stage; we 
saw several instances of users giving up on a promising technique or line 
of approach when tlieir initial attempts met with failure due to the 
unreliability of some system component. While the supervisor who monitored 
the experimental sessions w&s able to overcome the session-terminating 
aspects of most of the problems, there was a residue of problems that 
inhibited the effective use of the CONIT system by the EU's. 

CONIT already has a number of mechanisms that handle user mechanical 
errors and difficulties, with network and host systems. Considerably more 
could be done in these respects. The question of availability of systems 
and. rerouting is discussed more fully below in Section 4.3. 

< 
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4.1, 2 i 1 Ass i st aiice 

111 v'kmv ()(' the fact that oui- analysis has shown that search- 
strategy foi'imi 1 at i on is the critical area in wh i ch prov'^iess coii 1 d lead 
to improved retrieval el'I'ect iveness, we should comment on prospects for 
improvement in this area. Two pi'incipal approaches to makinj^ such 
improvements are seen at this poii^ ; (!) improving inst ruct ion to users 
on how to search effectively; and (2) developing additional techniques 
for automating search- strategy formulation and execution. * These two 
approaches may he viewed as symbolizing two alternate philosophies of 
assistance to users of interactive systems. Under the first philosophy, 
the user-is in cleai' control while the main finiction of the computer system 
is to provide instruction and marshal inf onnat'^on for the user so that he 
or she is assisted in making the decision, even down to deep levels of 
detail . ^' , 

The second philosophy, on the other hand, aims at keeping control as 
much as possible in the computer interface by automating decisions as well 
as execution of tasks; as a way to characterize these two philosophicfs , 
we may say that the fonuer emphasizes human intelligence, whereas the 
latter emphasizes artificial intelligence. In the information-retrieval 
application, we feel strongly that both approaches need to be used and 
integ^^ated in a coherent fashion in order to achieve an optimal overall 
system of user assistance. However, more dramatic improvements will result 
from new automated techniques. 

There are already some automated aids to searching in CONIT: e.g.^^ 
automatic truncation and basic index searching. |A number of procedures 



that CONIT now suggest sv to the user could be petf^ormed automatically. One 
example would be to perform automatically the hypto^hesized optimal stem- 
Boolean- intersection initial search based on a user-given phrase rather 
than forcing the user to do those implied operations himself. Another 
example would be to perform automatically a given search on several d;if- 
ferent databases v^ithout the user Jiaving to repeat the request for each 
database . 

I 
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vSt rat ejjies -tti her than the opvt iinal initial search can also bo 
autoinatocL DocidiiiK '^ow to do tins of'f'oc t j vo 1 y may rocjuii^^ additional 
investigations in sea rch- st ra tO|L;y theory, as suj;gested above. Meadow (1978) 
lias initiated one such invest Ijjat ion . One kind of :;tratef^y wovild foquiie 
a coinjuiter clustering and I'e levance feedback (see, e.g., Dosy.kocs 1978), 
r.ffective results from the keyword seai'ching in a (hUabase with cont rol led->. 
vocabulary indexing may depend on individual word |)05ting (see Section 
Jf -iaKlividual word posting is not impleinented, it may be possible lo achjOvc 
the desired effectiveness by a suitable mechaiiism for finding index phrases 
having a given word stem. We have j)roposod (P3) such a inecluuvism, which we 
have termed the Master Index and Thesaurus (MAIT) . The MAIT would include 
terms from all the inde?(e^^^of the various databases pJus posting iivfonn^- 
tion,>"rhis information w<^uJd be useful in selecting potentially releyaivt, 
databases as well as seai^oi^ terms. - 



A partial implementation of this? conCe])t has been developed by Battel le 
(Colombo and Neihoff^ 1977) for ''switching (controlled) vocabularies". 
-Other partial implementations aimed at the" dat aba.^e selection capability, 
have been accomplished as a research vehicle by Williams ami Preot^e (1977) 
and as an operational tool by the System Oevelopment Corporation (1978). 
The further development of MMT-like techniques could have a , crucial role 
in improved algorithms ior searching: 

In the experimental analysis described in previous sections^ we reported 
various instances in which instruction could be improved.' Desired imj:H^ove- 
inents I'ange from simple fixes in insti*uctional dialog to^ major modifications . 
iiwi-Tistructional fonnats^ media, dnd learning modes as discussed in Section 
3.5* 3. The online monitoring perfopned by the superviaor during the 
e)q>erimenta.l' s'^sions points to another mode of instruction that cOuld b^ 
highly effective: the online human consultant. This person could assist 
l^eyeral users .Simultaneously through an online dialog .which could be init- 
iated by either' a direct user request for help or by the consultant's mon~ 
itoring of^ many users and the observation of a situation in wliich consultation 
with the usei: could ^prov^e beneficial. 



As wr have said \\\ Section nu^ro important than thr fonnat or 
mode of the instruct io^i is its contoiit. Thus, perhaps th(^ most c T'itjca! 
tjuestion is f*ii\<.ling out what, in fact, makes for j.n)cHl search stratej^ies. 
Wo hypothesi7.e that the iwit ura 1 - 1 anguage-bascd keyword/stem approach is 
optimal for most searches as the initial scarcl^ strategy if one starts 
with a ^^reasonal) 1 y rockP^ set of keywords. This hypothesis luis received 
support from our experimen'^l analysis descrjl>ed Ikm^o as well a^^ previoi^s* 
analyses (see, e.g., Overhage and Kcintjes, 197^1). However, assuming 
that tfns hypothesis is verified, we are still left with important aspects 
of the criti.cal question to answer. llow do we find tlie'good keywords to 
start with?' llow do we identify those exceptional cases .in which the pro- 
posed stj^ategy is not optimal or even, no g6od at all? In these cases, 
and in the situations beyond initial searching where search refinement,^ 
desired, what . then is the optimal stVat(^.gy? 

The CONIT explanatory instruc4:ions available to users (see Appendix 

■ % ! , 

A) present some partial, tentative answers to these questions in the forjii ^ 
of various suggestions on searching in different situations. These sug~ 
gestions derive from work that we and others (see, e . g • ,^ Lancaster , 1973; 
Marcus, 1971; Jahoda, 1974; Oddy, 1977; Oldroyd, ^77) have done in anal- ^ 
y^^^ing searching in pnline systems. Howayer, much mor^ ^^^^'^ be done ' 
to develop those scattered guidelines into a .jcohefent theory, leading to 
optimal strategies for diverse situations. 

We can now see the broad outlines of such a^theory as mirrdred in 
the kinds of assistance, both automated and computer-assisted-instructional , 
that a sophisticated >interface would proylde. / r""^ 

4.-2 Interfac e Comp rehehsiveness \ 

The experimental CONIT sys\:em was designed with emphasis on perform^ 
ing in a common- language virtual -system mode all of the basic retrieval 
operations needed by the end vfeer to handle most of his needs. While we 



have demonstrated that this level oi' interracing does appear to provide a • 
definite utility for a clas.'i of users, a question for Aiturc research is 
'^^ ^'^^ specialized Tunctions can, and should, be incfuded in the 
coj^on. mode to accommodate ■ the" more advanced needs of users whether they 
arel^eg inner s or experts. 

y]^ .There are three considerations that enter into the determination 
of how comprehensive to make tffe common or virtual mode. The first con- 
sideration is how diffifiult -and costly is it to implement a. particular 
functlon in virtual mode. Some functions may^^ extremely difficult and ' 
costlr, if not impossible, to perf^.rm in. a given system. lixamples are search- 
.ing based on specified word separation in ORBIT or Ml-.DLlNi: and word search- 
ing of a nonfull-text indexed field in DIALOG/ A common approach can be 
set up for those systems that do implement these^nctions; to handle the 
impossibility of carrying out the given functa4«f^\;.the other systems the 
interface cQuld warn the user and, perhaps, suggest'--qr automatically exc- 
I cute"-substutite similar functions. On the other hand, users" can be forced^ 
to perform certain functions in the host (non-virtual') language through the 
pass- through mode. • , „ < 

The second consideration is the difficulty caused to users by these 
non-virtual situations. The third consideration is, simply, the benefit 
afforded to the uSer in implementing a particular function in virtual mode. 

Even if a function is ,/iiot implemented on a given . system , the^ inter- - 
face can, sometimes, be progr,4nmed to handle it. For example, CONIT keeps^ 
track of^thc current search set number and does not. require the user , to 
specify It even for output Requests to a system (DIALOG) that does impose 
that restriction. A second, jnove elaborate, example would be sorting output, 
say by author name. Whetl^'er to implement the operation at the interface 
to make up, for the defic^^ncy at the retrieval system is a question of cost-' 
benefits.'; It is clearly worth it for the first example (automatically' 
handling current set nu4bers); the second example (sorting) is more problem- 
atical . " c" /' .. ' 



4.2,1 Data St mc turos 

One area In wliich t!\c cxtciKVoii oi' virtual ity raises 
questions for tlie" direct ion of future interface development is the varia- 
l>i]!ty in bibliographic data structure among different databases. In 
previous reports (P3, P6) we described how this probl em ' impacts both search- 
ing and owrtput and the possibility of handling these problems through the 
mechanism of a common bibliographic data structure into which and from ^ 
which translations could be performed by the interface, analagous to the 
translation for the common -command language. 

In our research v^^found it possible ^ circumvefft this problem to 
a large extent by limiting the search and output options available to the ' - 
user in thevirtiial mode; in effect, CONTT presents a very simpler coimnon ^ 
bibliographic data structure and an acceptance of the lack of perfect trans- 
lation m some cases. For example, the QRBIT standard (defaql^t) PRINT out- 
put may not always have the same elements as the DIALoG mode 3 -^tput-~which 
we have taken as most closely approximating "standard" output. 

While ?uch approximation^ and limitations have not prevented the 

experimental interface from achieving a significant' level of utility, the 

question, of the value of the further extension of the con^hon bibliographic 

data st^nicture concept remains to be explored. Such explorations are 

needed to determine the feasibility of two major.enhajicements to interface 

capabilities: (1) merging results from seHrches on different databases, 

and ,(2) recreating database subsets at the interface for direct search "there, 

rather than* at remote host. ... 

i - \ , . . 

. 4.2.2 Databases and Systems 

Another facet of the problejij sOf differences among databases 
which we have anaXyzed with respect to our. expe\;riments is the variaton in 
search strategy ^^^uired to achieve satisfactory resulj^s in the ^dif ferent 
databases. .This is r'elated to the formulation of optimize search strategies 
as 'discussed above. ' ^ v f y 
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Another d iinei^sio!) lo the c'luost ioi^ of hitori^ico comprchons i voiios^; 
is the number oT dilTereiU retrieval systems to be accessil)le tluj^h^the 
ii^terface luMwork, Tl^e currciU CONIT system eonnects to three of the most 
widely used bibliographic retrieval systems. There are mai^y other such 
systems currently available which could add significant retrieval capability 
to ai^ Interface -faci 1 it'ated network. 

Oi'ir rule-based approach to ii^terface structure makes tl^e additioii 
of new systems to the network relat i vel y easy : basically, txu additional 
set of transVcition rules must be added for each new system-~the old rules 
remain unchanged. ^ In cases where the new systems shaix) a close relation- 
ship to onet already handled--^ . g . , NASA and liRDA UhCON are very similar 
to 1)1 ALOCi--rel atively few additional rules may be^needexl, (We note that 
ORBIT and the' two MI-DLINl- Systems already share many of the same rules.) 
In cases^ where the new system brings in diH'erent fuTu:tlonal capabilities, 
or handles 'a function in a very different way, relatively more new rules 
may ^be needed. The question in adding new systems js not so much of pos- 
sJJ)j_Ht7--that seems answerable in the affirmative- -but of practicality-- 
what are the %ost-benefits tradeoffs. 

Beyond purelyybibliographic retrieval there is the question of . . 
interfacing to darabases^of all kinds and to more general dat^ibase manage- 
ment systems (DBMS^) and, in fact, to information systems of various kinds. 
Some investigations have been begun in^this directioji (see, 'e . ^ . , Glaseman 
and Epstein, 1978; Erickson, et al, 1976'; SagAlowicz', 1977; Kameny, 1978). 
One eas^ii^that we have been able to demoi^strate the utility of the inter- ^ 
face ap)i^oach as' effectively as we have is that we have restricted ourselves 
to, the bjj^iographic retrieval application. It remains., to be seeiv how 
incrQasing generality of function affects the feasibility and practi-oality 
of achieving effective virtual-system .type networking of heterogeneous 
interactive systems through .the translating interface approach. 

••- . ■ /•-'•- 



We implemented our experimontaJ interface in a larjje time-sharing 
computer bec^^use this'^vas .an effective context for development, modification, 
aiKl testing. The best configuration for an operational interface system . 
needs to be detennined. A greater efficiency could probably be achieved by 
using a dedicated computer, assuming thai suc^ a computer can be kept rea- ' 
sonably luisy, A niunber of investigators (e.g., Anderson, 1976; Goldstein, 
1977; and Rosenthal, 197S) have experimented with minicomputer configura- 
tibns for interface systems. * , 

vSome of these investigators foresee the possibility of implementing 
interfaces on microprocessors and incorporating them directly in each 
terminal; such a conf'iguration has been called an -intelligent termin al. - 
The functional capabilities that are proposed for intelligent terminal 
interfaces do not, generally, come clos(5 to the range and depth of capa- 
bilities we have proposed fch* the virtual-system interface. Our CONIT 3 
system required on the oTOer of 2O0K bytes of storage in MULTICS and an . 
interface with a number of the more advanced capabilities -described above 
would demand more computational resources. Therefore, it is unlikely that 
a fui>^ virtual -system interface would 6e implemented on an intelligent 
terminal althoug h^some of the capabilities desired in such an interface 
could well be incinerated in an intelligent Hferminal. 

Because a virtual -system interface i|ivolves many scattered users 
and a number of scattered retrieval systems, the overall configuration 
involves a network and the network rajnif ications should be considered. In 
the first place, it is highly desirable that the interface and the disparate 
retrieval systems be embedded in host computers in a modern digital com- 
munications network so as to be atlle to take advantage of the speed, ef~ 
ficiency, and reliability such a configuration implies, in contrast with 
the much lesser capabilities inherent in the ad hoc configuration involv- 
ing the autocall unit we had installed in MULTICS, MULTICS i^Ninfact, a 
host computer on both the TELENKT and TYMNET computer networks on which 

many of the major retrieval systems, including the three accessible by CONIT, 
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arc hosts; howcvci', the sof^twarc and hardware facilities leciiiitctl to mixlv 
full computer- to-computer (as opposed to tei"mina 1 - 1 o-cominit erj connt^-t ions 
l^ive not yet been installed, and so we have not yet been able to take 
advantage of the improved communications capabilities implied by such 
])acket "Sw i t cbed networks. 

In an operational environment, it would be desirable to havy^i nuniher 
of interfaces d i st \ \ buted around the ne||W0!^k in order to i)rovide jiieater 
Y^'o 1 iability and efficiency in terms of load shariiig and reduced c6mmunica- 
t ions requirements. The ])Ossible design and con f i j^urat i on of such a dist.rib- 
uted interface system has been discussed in'^ie of our i)revions rei'^ort s (I'O) . 

4.4 Costs and Benefits ' . . ' ^ 

The direct benefits of an o})eratioual interface of the kind we have^ 
tested have beeii substantiated by the cxi)er imcn ts : simpler access to and use 
of existing heterogeneous bibliographic retrieval systems by all types of 
users, especially end users. Such systems could become accessible to\a large 
number of end users who do not now use them because of the awkwardness of 
enjjaging expert intermediaries to help tHem perform ^a search. [his, of 
course, may lead to reduced costs thrgj^gh economies of^ scale, as mentioned 
in Section 1 . ' ^'^ , • 

Although precise cost estimates cannot be madc^ without a detailed opera - 
tional design, some order-of-magnitude estimates^ can be made. The interface 
required duplication of certain functions regularly performed by retrieval 
systems: the parsing of input requests and the handling of dialog. Also, 
commuiri cat ions requirements are roughly doubled, in that the interface-to- 
retrieval-system links have to be added to the terminal~to~computer links. 
(It should be noted that these extra communications links, as we ^ave imple- 
mented them, generally do not add significantly to the perceived response - 
time for the user.) Some functions .su9h ^s selection of, and cofimand 
translations int9, target systems would be unique' to the interface. On 
the other hand, the major component function of the actual ' storage and re- 
trieval from large databases would'^not t\e required within an interf/ace, at 
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least within a relat lively modest s i 7.(h1 i nt (M'i\-u'o bi' the JejM'ee o,(' com 
plexity foutu! in our (experiment'*.. 'Yhc CONIT S \n\vv{':\rc systcMii THMiuireJ 
aboul 2{)()K bytes of stora}>,e in MllhTlCS. 

Summhij> up, we estimate an additional cost for t !k\ com|)Ut ei'- sys t em 
components of approximately tw6nty percent for this type of intet'face over 
those same costs for d i rect • access . I'he figure ol' 20 percent' increased 
computer costs is ptMtially supported by observations on cost.s of the exper- 
jmental CONIT of from to $20. 00 per hour. An operational system 

would be designed more \^ith etfjciency in mind, and could he expected to 
have lower costs ttian an experimental interface. Another benelVit of a seM- 
instructing interface is a reduction in costs for h'clpiiig new users; such 

help is a major cost in providing service 0!i retrieval systems. 

■if 

A more sophisticated interface with many of the advanced features des- 
cribed in this section would be correspondingly more costly, perhaps a-s 
costly as one of the retrieval systems itself. However, the benefits would 
include much improved retrieval culpability for all classes of users; it 
would be expected that a given retrieval effectiveness in terms of a 
certain recall level in a specified time would be achieved at reducx^d • 
cost through advanced interfaces. ^ 

It may be noted that certain apparent cost redlictions might be achiev- 
able vi^ith interface techniques beyond what has already been proposed here. 
One example would be the multiplexing of two -or more users on a single line 
to a ho.^t retrieval system; this would cut costs from the retrieval system 
by a factor of two or more at the expense of some additional complexity in 
the interface in keeping the searches of two or more users identified 
uniquely by user, and some increased response tinie yhen results are requested 
simyXtaneously. , Another example would be disconnecting retrieval systems 
when several minutes of user inactivity in making a request are expected; 
resumed connection and searching would be done only as requested by the 
user, arti past results would be automatically recreated quickly (without^ 

the time-consuming user typing times) as needed, 
] 

We have not included such techniques in our projection of cost savings 
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althouKh variations on them may j)resoiit poss i I) i ! i t ics i\)r savinj»s in 
user's tJ^JI]}^ because they represent savings that are more jii^^paront {hi\\\ 
real. J'Vho cost ''savinj^s" hei*e tlcpeiul on the f^^^i^ system charj^^Jnj^> by (he 
connect-hour , rather than total computet* resource usaj^^^e. livent ua I !y , we 
expect retrieval systems to charge more on the basis of total resources 

r. 

usee! (e.g., inchuling (^PLI time, amount of online communications, etc.) 
rattier than simply by connect time. When this kind of charging is put 
into effect, as is now do)ie in many cpmputer systems, i*esoinx'e -intensi ve 
search operati oit'^s , like truncation searching, will cost more which is 
consistent witli their greater effectiveness. This may make some intci^- 
face functions of the kind we have suggested more expensive in an inter- 
mediate future time. In the 1 onger-i^ange future, we can expect systems 
to be redesigned so tliat such desired functions will be clone more effici- 
ently for example, by stemming on input and Implicit storage of longer 
sets of document references ratlier than expl^icit storage. Improved, and 
possibly more standardi zed, retrieval systems may reduce the complexity 
needed in interfaces; in fact, new systenis should probably be designed 
with interface capabilities built in, ^ 

Without regard to the ultimate benefits of oper ational interface 
systems, there is a major utility to e xperi mental., interface systems as 
research tools . In this capcity it is possible, as we have deinonstVated , 
to evaluate potential modification to, or enhancements of, retrieval- 
system features, without requiring expensive and disruptive modificaAons 
to existing retrieval systems. Thus, for example, proposed command- 
language standards or search- strategy aids can be evaluated before great 
expense is involved in implementing wluit may be less-than-optimal 
techniques. 
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CONCIAISIONS 

In this report, we have describeci invest i ^',ati ons \uf^ ihc concept 
of a trans 1 at ing-coinputer-- int eiface/v i rtiia 1 sys tern mode for assisting 
users in tlieir access to, and^use of, lieterogcneous interactive biblio- 
graphic retrieval systems. An experimental interface system, based on 
this concept, !kis l^een !)uiU and ^vsted under contrc^lled conditions with 
six end users who had not previously used computer retrieval systems. 
A detailed analysis of tlie experimental usages fuis shown that the users 
were able to mastoid the ba^c commands of the interface sufficiently 
well to find aiseful document references on topics for which tliay had 
a cum^ent information need. 

The success of ttie experimental interface is attributable to a 
design emphasizing simplicity of use and to a comprehensive collection 
of instructional aids. An important component of the instruction involved 
procedures for developing search strategies based on a natural-language 
keyword/stem approach to searching. It is concluded that operational 
interfaces employing the limited set of techniques implemented^in this 
research can provide for increased usability of existing retrieval systems 
in a cost-effective manner. Such interfaces should be especially useful 
for inexperienced end fisers and- others who cannot easily avail themsQlves 
of expert searcher intermediaries. Furthermore, it is concluded that 
improved se^ch effectiveness for all classes of users is feasible with 
more advancjed interfaces that would include additional techniques which 
have been suggested by this research but not yet fully designed or 
tested in experimental interfaces. 
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APn-.NPlX A 
HXinANATORY MP-SSAGI-S RI-QUIISTABI,!; BY USI-.R . 

t - p^is appendix contains all the online instruct ion for CONIT 3 which 
i s request able online by the'tiser t!rrough the IU:LP and liXPtAlR commands. 
(Othei^ online instruction is ])rovided automatically by CONIT throughout 
the u:ser*s session; spe Sections 2 and 3 and Ap])Ondix B for details.) 

The foVniat of the explanations is that of the offline printed CONIT 
rcferenc.e manual. The reference manual wa:s not used by the cxperij;)cntal 
user in the experiments described in this report-; they relied on the online 
III:LP or KXPIAIN commands, or for Group A, the offline instruction \^hown in 
Appendix C. > - : 

^ The first explanation lists all oxplc^naflons available in CONIT 3 
■) ' ' ' 

and the KXPLAIN command that evokes each explanation. The explanation 

number is . not printed onliive. Thus, for example, EXPLANATION 5' is evoked. 

by the command ^'EXPLAIN CONIT^'^or CONIT" and begins as follows: v 

EXPLANATION OF, CON IT 
COl^HT is 'an experimerr^l 
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A f u H I i T 4 n ; of <J M th - C U M T xp I jnat i on s t hc»t yoj c.^n requtf s t on I i n« 
is given Dolo**, In ordtT tj soo on^ of thijse explanations online, type the 
the .iporouriit« l xPl A I coffliTi jnd t as sho-<n oo^oh. 1n«se »* xp t ana t i ons 
o'^rinte J" in >^jx»f* on 
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Mait for fr»v? w>or cje ( * »»»USc:k I I ) , <>r>J thon r-*itype the lirlxj. 
for more helo on ho« to us« CONIT, f y*?s« 
h^i Id 

tollowecJ by ^ Ccirri.>^^Q return; other wi^ien. you may nom typo. any CON II cow'tanJ. 
I«a0*^ [Time of disy y ' ' 

^ , — >■ 

THt t XPL A l.Nj XOMi AND 
The nane >f the oasicT^elp command explain . In usin>; it you «ay 

type eitho'^ the comoiete wond or i t abor ev i a t od for'* e . The comnand 
naiKe must always be tollOMej/i oy the itrfm you want explain and t>y a 
^ c^rr ii^y^^ return. For examoljdf if you typo . ^ ' 

expl 3in comrnands 

or last ' . ' 

.e com.T^ands . - . - 

CONIT will list al^ the bas i c >'c ommands you can use -ind,^^ J^ior t >^ 
explanation of tnem. Th0 inexpert user should -now typei e commands 

^tXPLA.><AT rON OF CONCEPTS * . . ' ' 

T6 have a concept explained type explain followed oy a concept name, 
So<pe concepts ,you c^n have explained are: " . 

' CONCEPT NAMF^ CONCERT ' \ 

commahds List ot CONIT commands \ 

converse ' ilechanics of the user's conv ersa t i orx > I diaio:| »liJi^t^ CON II ' 
conit , Short oacKground of -CONIT system 4|^- 

systems Info on retridvaM systems CONlT can communicate with ^ ^ ? 

data List of data oases you can search " ' ■ . 

for eacn ot tne above ex^lanaMons* CONIT wilt su^jgest o'thort more ^ 
de.tatled expi^ar^t io^ s that /oj can get. For a li^t of^aii available ^ ^ 
online expi 3naf5,ons that you can request* typei e explanations . , 

Not(j that, CONIT wiH aut >mat ica I I y (witnout ^our askif^g) <|i v e expi l^at i ons 
and suggestions during tt>e vioyrse of your, session if it detacts Errors' 
on your part or. as ,i t thinks that suggestions may lJ- helpful','^^ .Vs^^ 
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in t^r 
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n 9vi a 9 e of 
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c n r s t s to 

C I T * • anijuag 



4 f f er ef^ t 
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n if^ pr o v^r es>vj s . 
e t n.ind I all f un 
tho conne>;teJ r^t 
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arcn jrar^t from t 
t i on to s t uJy net 
or ma t 1 on s y s t o m s • 



ctions of ijvery system* 
r 1 ev H s ys t effif 
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he National Scienc?e 
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LX PLANA! ION OF HQH TJ OONv/EKSF. WITH CON 11 
You converse wit^ tONiT ay yivin^^ it comnanOs, Ljcn cowm^nci consists- of 3 
command name which'iay Oe followed oy one or more avUiTiooai Hords 
to maK^ he T^eafU ng o f the c ommand c I caH • . * 

To si^jnal tm? computer that you have completed yoar com'narto you HUST 
strike the c^rria^e return Key; /he computer i*l I no r^js pona until y 

CONIT Fespo-^d to your commanJ hi th a ipe^sayj. To'si^^nal th<>t 

its message is complete ana that it is a-^ain waiting for your command 
^CONIT Hill orint the •'user cue* t •••••uSERti 

Y^ou cannot ^iye a commanJ until you ^e t the OStriti cu« au t you can 
interrupt CO>liT in its processina of its' last command by typinv>/ the 
eKEAX Key. CJNIT Hill ,theni :?ive y6u a l|SEKi i cue. ^ ^ " 

Fbr most errors you maKe CONIT will ^ive you an informativa ^rror 
message. If you detect an erf^or BEFO^C: you stride tn<, carriage return, 
yoj can cancel nhat you havt typed so far by stri»<invj t ne liKtAK Key. 
receiving t^^ JSEK cue you 'm^y rety6|> the command carreqtly.^ 
Uf you ^Vconhectea to MUlTICS vi a ARPAr^ti I , sena ^t<L^< tij typin^t 4 
for information on other nay^ to edit typing errors, tyoef o eo i t 
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tXPLANATlJN Of- COKKtCl I r4G -Ct^KOKS 
or most errors you maKe CONIT m&M i v ^ you in info 
me^m^xs. If you Jetect an error BtFJr<L you striKe tn<e c 
y ou cc*n one i I wrij^ t ydu /ijv ^ ty p «(J '^o t rir t>y s t r i K irvj t 
r^ceivoin:! the USER cu« you m,iy retype trie commcnj ^orr 
<lf you ari? conr^ected to riUL 1 IC b» vi3 AKPANt T , seno Jf^tA 
Thie BKl A^ Key l^omvtimos I at) ^ f eel ATTL.^^flON (ATTN)) c 
Interrupt 1 I .actions AFTtK you request them; for ini 

1 her o ti«o ways '^to c<*j>cel error*; without hawiov^ t 

1- Type the AT SIGNMo)) to cjnqel tne comnicind' I t ne up t 
immediately typi» the correct line, lhus» if yoXJ .^P^ 

s^hoe dt^shoH dat d • " 

you Hill iorrectly '^af the :>HOW UAIA command, 

2- n NUMtiEK <i) will cancel the last n ch.iracters 
Thu-, t either of The two lines 

snoefw lata i 

shoe daiifiw data " a / 
will correctly enter tho SHOW L)«A command. 

shoe d#ifw daat^shop djt^dsj^w datola 
M you c^re connectei MULflCS thru A^^r^ANLT, you must 



Comoi n^i 




r m at i ve err or 
arri *9e return ,Keyt 
he i^KtAK Key. After 
oc 1 1 y • 

K by typ in^ i iis i p ) 
30 al\»o^oe used to 
o tyijeJ e interrupt 
o liJit. AK^ and .w«i i t i 
o thit point and 



you typedt 



> a I s o w or K ; e • y • t - 
type 2 AT 5IG.^S 



EXPLANATION Of^ INTtRKUPTING ^ . 

The ORtAK Key (also callt«d ATTEf^T.IUN (ATTN! can oe ^sed to interrupt 
actions and messa^e^ oefore or a^ter you request them Oy 

simp'ly j^pressing >ne BREAK Key ONCE aod i^^aitirfsi for CONIT to »^ive you t^e 
US£Kti cue oefore maHiofi a n e w r eques t . . T hi s w i I I /^t jHe ^ variable amount, 
of time dependln^j on how loncj it takes ujpNi T to ^et the oth-^r sy^;tem 
t o jS toptalking** ° ^ 

Ihi s fcatjre ma/v be v«ry use^fui to avoia waits wnere« <£*si*» vou have 
afc^r^dy seen some instructional i^foi or you hayj^'.s^en onoW jtT documen t 

er^n c(* outpu t , or you ^re simply tir.?d of Waiting for an action to 
^^tart'pr be/ c omf>l eted. ^ ' 

INOJEI BREAKING occasionally causes systems to drop 
llf you are connected to «UL T I^^S t nru 4'><f*ANE T , send 

.• ( - ' 
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ou^ or hranv^ up • ) 
dREAK by tyoingri 
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:0N 1 I t X f^L A.>|A T 1 UN S 



' tX PL AN AT ION UK COMMA NO j 
f he p3';4C commands you will oe giving jr^ 
dt^or ev I t* 1 1 on m ly oe fyoed)!-** 



full Hfi me or 

NAliL A3t3<r\/ 
• xp I air^ L» 
p i C K O 

show s 
coflioine / c 

s too 

in d tJd 1 1 I on 



S-IOKT 



Used 
Use J 
Used 
Useoi 
UseJ 

f o fri e 



.to 
to 
to 



t KPl ANA TION 

get exp I 3nd t i ons .iu ou t uii n^j *CO'N I F 
plcK a data oaso (,ino retri-jvjl system) 
find documents in a d3tn oasv* 
show informvition on documen f s t d^t bases* 



etc. 



sp«c i a t 

NAMe 
us«r cue 
RETUKN 

BREAK 



«rr or 



to conTTiAe sets, of retrieved Jocu«^vints 

to inaKe a <^|^iii<»nt or send command iVs non-CJWlT I i#nyoiago 
to stop your COt^IlT -session 
ne^^ar cowmands listed aoove, #*e not-O oe I om 



are* iioortant to the 



a f eH 

di a I o^ I 



th^ user may type 
the Ke TURN Key 

• ) t o ^Cfk»ce I a 



operations oy C^NIT or the user which 
OPttiAri ON/MtSSAGE ; ' ^ 

J^Mi String USbR M • is CONIfs cue tnat 

At the e'ld of eaotA command the jser must typ 

to si^oAj the computer to do the co'mip^inCI 
Strike tie ffREAK KEY O*^ NOT tyoe the word 'oreaK 
typed t i ne or t oi n t err up t CON I T 4 r*a it ' f or t he. 
oe f ore t yp i i ne w^^l i ne ^ ^ 

f3r a detailed explanation of how to correct typing errors 
(with or without usimg the liRt AK K^> • tyooi er errors 
M^e det.^iled exolanjtions of the' above commands and operatibns can be 
hacrby itypinj explain or ^» followed by the name of the item you 

wiS?h tj^ have ^^lai->ed and ^ carriage return. Ot^er itoms yop can tiave 
explained are listed under e concepts . . . - ^ v ^ 

The itltt^barj uier should now type! 
e p icH I ' . ^ 




r 
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COixIT EXPLANATION^ U)-L1 - . 

• M U ■ ' . 

OAfA liAS.. ^oJt.:! AKtAS 
L I t e J o o • ow jr ^ 1 su t) J t t*r o «ii in >«<r» i c n ci j t j t) j ^>os .irt* av a i I jo t** 
I . P h / \ i c 3 I S c I -> c e s < Ph y s 4 c s ♦ r. h efp i t r V ) 

<d . L r>v|i jv;r i n. ^ (inclvjdin^ Ct^emicjl F n ^ i « *fr i Ovj vjn J p,>tor>ts> 

J. b eo ic i eoc 0 > 1 :.neVviy, * nvironmoat ' x 

-4 • (j 1 om*> J i c«> I f . A cu I fur e f F ooJ 
. Social Sciences*' Politics* tdvjcotiont Mum.4niti^*^ 

b. 3 vjs 1 '>L* 'liar^ « t i , ianagemer^t* i^r^ints 

7. i^eneraJ. laM* or Mo^t* s\jt)|ect- areas coverei) ' 
For a list ot the dat^a yaso* in on^L? of tnosi? areas, typ'.^i Sf>ow Jata.X " 
when 4? X tn»*. nvimOor o t t r>o ar e a ? • sJ • » 

OH c1 it a ? , - 

will list t h « L n i n ? er I n g cJ a t a b a s « s Y ou c a n ' s If ir c h . ■ 

OATA 3ASLS IN PHYSICAL SCltNCES, . ^ . ' \ 

For «dct^ Jcjt a ba:;o t t>ere i s I i stoJJ 

CONIT njiwDtfr ^ 
C J NI f r> *n>e 

code lv?tters of system.; naviny tr>e data base « 

ID^OIALJC., N:=NLM/MLUi-lNt T 0=ORfiIT* S= SUNY/MtOL IN^. I 
* sh or.t xpl ar^a t i on o t 0 i t a base ' ^ . 

.11 SCI (0). Sci f>nce ^Citation -l^ijXlex ( SC ISL AkCH) * ■ 
12 PHYSMO) Science ADs"trWas-A« Physics (INSPtC) - ^ 
Hia CHEH7i^ lOJ Chemical Abstr^ac-ts Condensates 1^77- (fCMtMIw- inJexir^g) 
13b CHtM72 CJ,0) Ctiemical Ao s tr ac t s Condens a t e s l^/^?-7o <1972- for O-^BIT) 
l,3c CHEM70 IJ,1)) Chemical Abstracts Condensates* ICH^^flCO*^) l-:*70-71 / 
CHtHL^^ li)J Chemc iai Abs tr ac ts SuDlect Index Albert (CASIA) I^J-tb) 
^I3t CHtMTt^H ID) Chemical Name OicTionaryi CH^flNAMt; "se.; al>o Ht OC^tl TE «.i t hJ 
You may now pick a data o a'se dy tvypin^ p X ••nere X i r. , a nVjrtoer or name 
However, it you need more Information on a'Jata DaSo before picKing,^ ei tier 
refer* to the or'inted descriptions, if |yau have them, or^ typ-ji e data X Y 
Nhere X iV^^^e data case number or name, and Y is a syst^im name. t**i«» 

e dat a 1 Jb oroi t , . * 

or 1 ' 

I A» data c hern/ ^ dialog ^ • / 

Fo> a I ist .of deta bases in another of th^^ ieyen aroas» use tne 
SHOW DATA command ajain • 
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t XPLA iA T I O.N 
\ 



JAIA 1>ASLS In i>*C> iNLf KI nG 
for vacn i<itj t>a»t» there is 
CJuLT r^jmo*»r . 



y 



coo^ lottery of ^^v\.tefn> ht^Vjin^ th« J.n^* b^^vt) 



snort \*Kpl anot i 6n of(1*tcit>asc 
tM^lN t:)fO^) tn^^ineering Inoox (COMPtiM0_x 
EE ( J ) S enc e A o s tr oc f s ~t)» C ; L I i>c t r i c<i I 
Mt' 10 1 I n f or mat I on :>ery i c e in Mochani c I 
MtTALS (:)J Metdls A t) s tr .i^ ts / A M o y u [na*'x 
ALUM lOJ WorlJ AlumiooM Aostract^i (,^AA> 
PA T Ut^^^l J J , t « Cen t r Jt /Hor I J P > t ent 

PAflFI I.)) lKl/P>lonuml J'^nwrol, electric 
PATTtr^.i IJl U.S. Patent Office Classific 
PATCHCrt// ( 0 1 IF i/PI enun t Ct^emical Paton 
PATCHtM>U (6j Serine as PaTCMfcM77 for 19jO 
PATCHLiCA [01 Cr»emical Aostr^^cts Patent , 
PAPEK fJ) Pao«r Cf^emistry 
YoiJ «ay noK Pick a data oaso by typir^g 
Honever » if you neeJ more information o^ ^ ^ 
r«fer to the urintaa descrio t ionsf if y.^ovj n^i 
Mhere X i .i tnt* data base nu^iber or name an.J 

eaata.!lorpit \ 
or ^ ' I ■ . ; 

e data ^ng in^di a 109 ' 
For a list of data Dases in another of t-ie s 
SHOW DATA command a^ain. 



Z2 
23 

2*>a 
25d 
2'St 
2ba 
26o 
2bc 
2 6p 



, t i > 

,i I *?ctrvOnic tComput«*r ,Cont ro I 
L n 9 i n o i r I n >i ( 1 S Mk C > 
(Mt TAO/.XI 

s indexes 

alf ana mechanical u*S. Patents 

at i on Codes ^ 

ts- U.S. an^J forei^jn equi v (1^7/-) 

-7b 

C j^V^cora jncir t US and f orei gn 

\ * 
p ^ wnere X iv> a number or name. 

atii base before piCKinjt eitnor 

ve \themf o^ typi^'J < data X Y 

Y i '?> a y s t t» m name, t . . » / 



eN^^n ar^jas, use the 
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DAIA UA>tS lN\,LOSCUNCLSt tNt.^bYt |- .^V 1 f^O N.it N I 
f^or eticn Jjfj ba>v.> there is Mstvjcil 

Ci) N I T r^vjinb v.t / 
COM I. name 

cooe l-jtt«rs of systens havin^j the dJtj jtj<ise 

[O'OiALOGf N=»>4LM/MEt)L INt- ♦ 0 = 0^011 , S ^ SU N Y / Mt .) t 1 N c ) 
snort i>xpt liOi^t i on of d<<ta oase 
II SCI 10) ^icience Cifition Inoex (SCl^LA^CtU 

HE (0 1 Ifitordjtion Servicv? in Muchtioic^^^i^ trwino^rin^ tlj»MtC) 
ii! MtTtOK [J] lie f eor ol og ic J I ^no G t5ot>hv s i c i I Aostrjcts ( i^GA ) 
^ OCt AN 10) Oceanic'^Abitr jc ts 
^ 3Sty AQUA 10) Aquatic Sciences .ind ^ir^hories Abstract .* (ASfA) 
t.^yiH 111 tnviroonient 5l Aoitracts (tNVl r^OLI fC) 
i5a POlLUT (0,0) Pollution Ao^itracts 
35b AIKPOLLUf lOI Air Pollution <APTIC) 

36 ENtRGY (i)»0) wn^rgv 1 nf omat i on and Lnvironment Aostracts <LNt«bYLlNL) 
3/a OIL (0) Oil ana 9a xp I ortJ t I on t ct«i vi? I opmen t ^n J production (fULSA) 
37w OILNEWS 10) Oil and energy news (P/fc .^S--Am . »^etroleum In'ititi^te) 

You may now picK a dat:» oase Dy t y 0 in^ p X 'wn ere X Is a numt)^ or n amv* • 
However ♦ if you neevi more i or ma t i on on a dat<* Dasj pot ore picHinyf either 
refer fo the printed descr i o t ions>f if you nave tnem, or t ype I e data X Y 
where X i^s tne oa^ta case nunber or name ana Y is <^ systo* n.*«e. t.g.* 

e d a t a 3 b a o r o i t, 
or . " ' 

edatapollutdiatog 
For a list of data oa.ses^ in anotj^er of the seven arcast use the 
SHOW Oft^]^, command ayain. 
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• • 1 i* 

illOMt. 1)K:AL t AC»^ ICUL TOK^L. ANO e JOO JATA (U ^.>t: S 
Kor yjc^ lot ii o«<se t here i s I i s t i; J ^ 
CO Nl 1 nonu er 

CO M I n.jmo < 

I terv. of sy'> t«« > ha v in^ the J.Jt .1 u«ise 
li)n)lAL00t *^NLM/MlOe IN^ • O^^ORBlf, ^ = SU N Y / ML l)L I NM 
short vjx pi, loot ion of ditd oaso 
11 SCI 1 1) ) j»cienci? Citjtiof^ Ind*?x (SCI>lA<CHJ 

AUKlUS IDfO) U.:>. Nritionol A <jr i c w I t vjr j I Library (AGfr^ICOLA> 
A(»RlCu^ (i)) Com-nooMeal tr> Agricultural iJurtiiius (CAO) 
4»lc FOOD (DtO) i ood Science anJ Tcchnol 097 Abstr ac t (FSIA) 
<»lr AGf^lRtS f:)J A^^ricul tural r<i's4arch ( USOA CR I S ) 

€10SIS IJtOJ Biolo^^ical Abs . an .1 . 0 i or es« ar c h Iniex 147^- (0^<iUTl 1^7^-) 
<»2b 0106^ 10) 0 iOSl i 196^-7 J 

kZf blOTtKrt lo) (HOSIS codes af>J ^ynony^s dictionary 

h3 HLOLIUE, ( StN] l-tdex rtedicus, by Nat i ona | L ibrar y of (i^icin<? ( NLrt ) 
Mt^OMON iNj .>iLM-c atal o^eJ monoyriphs (OArviNE.) > 

<»3t flPoThKrt [ s^jJii^Medical Suojocts Headings piction*ry (MtSH) 
43v MEJAV [Hj rtedical Audio-Visual materials tAVLINt.) 
i^k TOX CN) roxicolJ^jy (TOXi^lNEJ 

i*t*d TOXtFF fN) foxicity effects datd ( NI OSr* - KTt CS ) 
<»^t TOXIEf^.i lU] Toxi col ogicv* I t er»s (lOSCA) 
k5 t**IL IN) tP^lLtPSYLINt 

CANCEc^ l^l) Cancer Thera;>y and Carcinogenesis Aostracts <CANCE«L1!) 
^/f^CANCEKKES IN) Cancer re^earct^ projects (CANCtr^Pr^OJ) 

You way noN pick a data dase by typing p X ••nere X is a number or name. 
How«vert if you neeJ more information on a data oas^ before pickina* •if'^er 
r«f«r to the printed descriptions! if you have themt or typei e data X Y 
Mhere X is the data oase number' or naiwe and Y is a ^ystrfw ni«e# E#g«f 

e dat a <♦ la oroi t 
or - 

• data a^rius dialog 
For a list of data aases in anottier of tn€ seven ar^^ast use the. 
SHON DATA co««and ajain. 



<»3fYJ^UN0H f.^fN) Current mof^th of MtOLlNt ^ v a 

Hi5^lfcjGHL^rt:Kf1 (S,^l Cheinical anJ .iedical terms (CHEMLINE) 
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JA TA liAiL^ IN . SOC 1 AL ISCIl.NCt St POL IT I v..; , K OUCA I I ON , hU lANl F ILS 
For vacr* Aa\ ^ t>a »** Thert? i <; listL'cJi ^ » I ' 

CONIT r>vimu»»r 
^ . CO Nl I n ime \ 

coo« Itjtters of syitems having) the cJat** base 

ll)-01AL0(^, iM=NLM/MLOL INt, f 0 = Jf<On, S = SU NY / rlL 0 L 1 Nl ) 
snort k>xf>K*nation of cJ<4ta base 

31 SOCSCI I i) ) Sociil Sciences Citation In J x j^Oi^SC I St ARCH> 

32 .SOC CD) ioc i ol ou| ici* I Abstracts 

'^2b CrULUAD ( f) J Child abuse and neglect 

PSYCM C.)} Psychological Abstracts 
$<»a FtO 101 FtOERAL INDtX to (jovernment activities 

S%o,^gKtCOKO' 1 01 Congress i on.* I Record . - » " 

5Uc PUiiLlC Ivll Public Affairs I n f or «a t i on S er v i c « (f*AlS) 

5<»n FtONOW 101 Most recent month of FbO records ObFJKt loadin^i into F tU 

LAf^G ID) Language and Lan^Age lieh^ior Austracts (LLiiAi 
!>5b LIBINFO COl Library and Information AbstractJ^" *IlJL^* 
^6a £0 (OtO) Educational Resources Information Ceaftfrll IC) 
56b ElkxCHlLO lOJ LKCeptional 9hi f d tducation Aostrjcts 
56c EDAV 10) CurriCvifum planning media tateriats (NlCtlMlv 
56d VOTtCH (01 Vocational and technical education \ 
57a HIST CO) Historical Abstracts < non U^S.l J 
5/b AMhIsT (□) A*mericax History and Life'<ArtL>f U,S. and Canada 
5ft ART 10) Modern art and design \ 

You may now picK a data oase by typing p X . wnere X is a number or nawe. 
HoMever^f ifXyou ne^J J^oi*-* information ort a data oaso before picking* either ^ 
rmfer to t h^s^r i nted' descr ip t i onst if Vou r^ave them^ or t ypvj I e data X Y 
Mhere X is the data base number or name and Y is a systx*m name. E.g-t 

e data 5oa orbi t 
or ■ ^ 

e d'ata ed dialog^^r* 
For a list of d a t a^a^<^^> in another of tne seven areast us« the 
SHOW DATA command agaflln. ^ 

■ • -i^f ' 

>^ rhis pa<|« last up.istedt ^7-12-17 
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OA T A 3A ^t:S IN. 3 
For e^cr» data ods 
CONl 1 nvjV) «r 
CONlf niwW 
c (> o e I ^ t t o f s o f 



CON 1 1 t Xf*L A 1 I ON {^h 



t here 



MA^^Kl 1 INC, 
is ! i s t u 0 : 



i 



MASAULHr.Nl t OKAi^il S 



fvi V i n ^ the 



system; 

lO^^OlALOb, N=NLM/Mt.Oi.ir* , 0-Or<on 
snort oxpl jnation of cJitJ D<iS\; 
BOS lUt 3) >*iusines^, m3n.igef«*»nt ♦ iwarKetiny, vjtc 
OJSCAN tJ) C>if\aJi3n Business Periodical Ipdex 
b^a M4NAGL lOJ MjOcj-^tment, f 
hZv At^COUNJANT [0) Accounti 
bTi* MAKK£WU.lO} PK-OICASTS 
hS<S MARKElOOO [Ul t^overnme 

MAKKeilN to) PKE0ICAST5I 



/ 



6lt> 



bii 
6 3n 

bSo 
65c 
66a 




< Ain/INFJKM» 

lance f ^ndvj^ tr i ai re I at i onst etc 

auditing! Invjestwent :>f tax^tioof 
WortdMide cotuDany aad .iroJuct info 



contracTsA KF^*•s» etc in aefense areas 

Indexe'i to MA^KtTAH 

MAKK£TNO^ COI Cvjrr^nt «onth records ^EFOKf. /I oad iny into (lAKKtTAO 
STATPLANT lOJ fclSl Info on businesi es tdH i sh«ent s 

STATUS (0) PRtOICASTSJ d.S. indvisTry and yoverntient statistics 
STATUl^TIrtt: [01 PRtOICASrsi ArtnualMiine series for U.S. i^nduitry 
STATRtG 101 PKtOICASTSi Mme Series tor U.S. metropolitan areas^ 
SIATFRN I J) S^tnilar to STATUS-for for^i^n iitati>tics ^ . ^ 

STA„TF^<NT iMt tOJ Similar \o SATUSTlrtE for foreign statistics 
GRANTS 10 1 Grant programs by »^o v eVnment a I , com.^ercialt c*nd privat 
FONOIK IJ) FOUNOAllON OlRt-CTORY fnon- jov erhment 3 1 » nonprofit)^ • 
FDNG>«^^ Grants ^iven Py I ovjnda't ions in F0>l01R > 

CrtEMNtWS (0»0^ News from Chemical Industry Notijs oy Chem Abstract 
6bb ORUGNLWS (O.OJ Phar maceu t ic^ I ,>leAs Index - . 

You may nan picK a data D<*seSy t^pin^ p X . nnere I is a nunDer 
Honever. if you need more infornation on a data oasa pefore pic«in^t 
refer to the printed descriptions, if you r^awe them, or typet e data 
when* X is the data base number or name and Y is a^syst.em name. t.9.» 

Q data 61 orbit . ' 

or . 

e data ous^ialoji 
For a I ist of daia baits m 
SHOW DATA command ajain. 

; 



e 9rovips 

/* , 

s 

\ 

or name 
ei tr\er 
X Y 



another of tne seven areas, use the 
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JAJA tU'^t S COVlRINO ail, MOST, iUJJLf^ ARt A . 

f',or h i J t J o <j s « t n er e is I t s t o cJ I 

CO HI f ovjmt) <»r ' . / 

CC5NM r\\m^ i, 
' caae letters of system.* having th« <Jat ii l)'lt?f 

10 DlALOt., N=NlH/MLOi. lf>lt » 0=Or<UI T , ^ SON Y /ML Jt I NL J 
snort •^KpKination of d^ita oas* ^' 
NTl^ [OtJl Governmen t-'saonsoreiG KIO reports <Natl T.jcfi info S*^rvict*> 
L lo CO.^L 10} Li D^br y of Cong r ess Oooks j nu rion<^r aph sintn^lish- >, 
LliJCONF'CO) Liorary of Congress foreign < oon-Cn ^ I Isti) m.*t«rlafi 
SSIL (01 Smithsonian Science Information f.xchan^e (current r«s«ar.ch) 
TMLShS (0) ComprenensiviJ Oi s s ir t at i on Aostracts (ooctorjl theses)^ 
NtWSCAN (0) Can3dian Ne'ws Indfex 
You m.iy noy pick a dat.j oase 4y 'typing p X nnere X 1 s a number or nart>- 
HoMover, i f yjftu neeJ more information on a data Dase ftefore picKlng* ei t^^er 
refer to th»? printed Ue scr i p t l.o ns , if you nave them, oV typ*jt e data X Y 
where X is the Jata hase number or name an:1 Y is-.^ system name. E.g., 



71 

/^e 

7^f 

' 75 



e (I3t 3 <^ 1 orbit 



or 



e data ntis dial 09 
for a list of data oases 
SHOW 1)ATA command again. 



in ano 



t rier of 1" n« evien ar \j as , u se the 



\ 



EXPLAINING OATA BASES 

In* order to get an online explanation of a data oase, 
Hhere X is the CONII 'number or name of tV»e data oase,. 

A given data oase may exist on two or more syste-^s^t in 
the imttlement at ions and explanations nil I oe different, 
the explanation for a particular system, aod the system 

e^da^a ed dialog 



typet 'e 



data 



wn Ich 

it you . 
name to 



case 
want 

your 



to ge t 
command . 



th4 



eo data base fro» ^lALOU and yKttIT\ -^resp ec ti v 
CONIT will.picK^ne for youT^as in t^e PICK 



and 

e dat a ed orbi t ^ * 
Mill get explanations of 
If you do not name \^yste« 

JoMmand . - ' 

If .^ou ara'not connected to a system from which you want an explanations 
Ory.? wiirconnect you'<after d 4 sconnec t i n>.-iyom the curr en t ^y s tam and 
dirf'a Dase» if any). The databas^e you ar as c urr ent I y connec^t^d. t o Mill not 
.br^^hanoad when you asK for 
,is f rom VaSsystea otr^ar than 
' case you' wi I I oc.-^oYnec'ted 

EXPLANATION OF )F.ItLO» y . 

^ Fields are c ategor ies /of i nf araa t i onv.in a data oa^a (e*g.t titla* 
Cauthor. docMT^ef^t source / report numoer, Jtidex termi^ :^tc. I that , can ,De used 
for search^i or fo'* s^bMiog JocuJfents found a^ the resliVr:c>f searchiny.^^ 
For simplej^earching ^FlNdingl ^nd :iHOWing-ir^ the I an^ juagc 

you do nol4r»«^ ^^o** field naaas ^f <the datdi bas-'S, out 
for specialized, searching or shoMing^-Mtiich must ue done >n tne .language 
the retri*\/al system, you must use the data t>ase 



^ an explanation, unless t^e e^kpl anaf ion you request 

tt>^ one you are currently connected- to. In that 
to the default data base of tne expl ainl og s/st ^m. 




FI£H)S command af te*" the t 



done /n tne 
field names 
i)A T A- command. 



•l2-^l 



9.' 



'I 
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CON I } c<f^L AflAT I OH ^0 



t Xt^LA.^^At ION OF RICK CO^rtANO 

. I h« PlL'< com man J i :> used to plc^*" » djfa oaso to ioarch in. 
To oic»\ a * I' isct type picH X or simply p \ . - 

wher« X is the numoi?r o^ na:Tv« o"f ^he data Dise; f or-^ ox *)«p I v* i 

p i C K ^ 3 • - - ^ 

/or „ - 

. p meo I I X 
NwiJI^ijet yoj connected to th<» Me/OLlNt dita oase. F,)r an 6nlir)e list, of 
v'T.he datf^bases you may PicH» type . ' 

* show'dat 3 ' ' s > , / 

To switch from o^e data oase to jnotfWr, t^pe ^icH Y Hhere Y is th« 
numo**r 'or'^n^me o> the new data oose'. NaV(*» how.*ver, tha t sh i t c hi n y y en*r a M y 
results- In previous results oeiny dropped oy (he con«put«r.^ ^ ' 

^^In or^^r to save on c onnoc t- t i charges you may, at ti«*iSf want to 
disconnect the current datc^ bAse; to do this* type» 
di so onn ec t ^ • 



, 9 



A 1 1 <?f*^ d I sc onn ec t i n ^ 



orbits abcr ev i a t i oi^ 

■ clis^ - ^ /j • V 

, Againt di sconnpc> in^ sjener at--Hr"f'^*J s pre\; i ous ^esu It s . 
vy-ou must Pr%< again ^o resume searching. 

^^j;i)NlT will automat Ccal ly connect to a system tnat ha^ the data base 
you picKeJ. For an explanation of how you can override CONIT's choice 
of systeWf typei e picK system 

II the aoove expl ana> i ons ^ e on "paLper^ w« sugs*«>t toat the inexpert u^er 
now pull up the paper* tear off the instructions for reference, and tfitn^icK 
a data ba%e (after getting infJrmfttion on the data-oases, i t •-n#ede'0) • 
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tXf>LANftr;jN Of SYSTEMS 
CUNll 0:*n c^^nrVecf^to S Jifferent rotriev»il ^ys1^n\>t i) Mi()Lll4t of 
t f>e Njt loncjf tLiorary of Mediclni?; ^) LocHheoo OIAlO^ (l-^S); .lOJ, J) Syst^'* 
•tiovtOoumont C'or cor3;t I on (SUC) 'JKBll . / 

'^hVjI- 1 whosv main- Jata oai>e is also n«inieJ MtULlutt spec:iali2«s in 

mjft.dix:i'^'» e. It also has other data bises in relat^tJ oiomoJicjl TielJs." 
Botn JjALOJr ioo OkDl 1 have many oat3 ba^ias coverinv^.aM disciplines. 

Kor a li->t of aat a b ase^ava i f at> io thvj differ ont syst««s, typoJ s data 
If one syston^ is not .ivaLi^t>l«« or to msKij morv complwtr si?irch\;s, 
yoo ma/ want to try s»v^3l systems and data oases. 

There cfr^ two verslor^s of the MtOLINi svstt»ft NLM .^nM SONY. lh« NL M 
version h^s j f i»h d.^ta bastes. not availaofe at SUf^V . For mor-i info picK ^ 
a third systvifnl NLM NEWS* »4nen you/piCK me^lino* CqNiI tries to cjet SUNY. 
Yovj may ^let NU^ by typing* oicK nedline nl« 

i;QNIt uses etthen the lf.Lt-«>ltT or TYMNtT networn to get .a system* 
If a'systen Is not availablo tnru one n.»t^*<orK, you mjy try the other; e.g^t 
pi cK mo j » i ne t e I en e t ! 



or 



•Pi ck" m^id I i nc nl m t e I enf; t 



hXPLANAMON or PiCKlN^i SY:> Tt MS 
To connect to a .1at« ©aso COMir must^first conn<*ct tj a retrieval 
system that nas the data oaso. * Spme data ^ases are available on more 
Than one system. CONIT w i I y au ta4a t i c a M y select a system for you 
unless you prefer' to override CONIfs chotca. To do this, specify ■ 
ooth data oase AND system? f or 'e/x amo J e , 

piCK ntis or.b i t ^ 
connects you to. the NTIS 0ata b/ase in OKOIl, whtlt> 

p^ldialog' /. . . 

connects you to the NTlS data/base, in the JIALOG systei^ 

•If you specMv ^ system oCjJi nd data base, you will uV.connecled to 
the default data base for thjkt system* Fof example, thJ oefault data bas^ 
for DIALOG is El)» / " i 1 i 

For more information atyo^t the sy s t e-ns " .>nd net wor K s^^Jlj^^* t e systems 

SHOWING STATUS . / - » . . 

Tt>« SHOW .STATUS command tells whjt dal^i odr»e and system vou are currently 
connected tOr If any. " /. 



LXPLANATIOU OK 0 1 SC Oj>*»Nt C T 1 iNb 

rn^ UISCONNlCT comTiaTncI ^ oor ev i a t J on •disM can us.eo io disconnect 
the currently connected r e tr i ev a 1^ sy *i t eni <ip(J Jdta o.iS*i. kI.II ^eay« 

yoa connects* to"CO>iIT oot Mill avOi^ aciy jJditional ch^r f r oift the 

rvtrivval systems. .Disconnecting w4ll.<Jr/)p from th> compotwr any <;»^rreot 
search sets (offline requests Mill NOlr^b^ at I ec teo, hON%jV er > . 

To cJiscDHOtJCt from CONH (as well as .any c vjrr en t r e t r i «y i I system)* 
use the STOP cofuraanJ. 



XPLA.^AT JLON or STOP CO-iMANO 
The command 'stoo* Hill t off CONIt at t «r 
any system -tnat yoo may currently De connected 

■ ieVore you s^op CONIT He would aoorec^^ate 
For explanation type'i e comment • 

You may disconnect a retrieval system HlfHOUr 




jdi stonne.c t ino 



omrtents you may have. 



stoppin^i CONIT, typel e d 
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CO<iHANO 

s<t <ir c h f or j oc u«%!n f ^ i nJ x? x a^J 
'wh«r'0 )C i s thq t«rm you «#r « v 
^ptcific to your-toolc. I no 



that tjr© 



I X»>L A^Ar ION 0^ F INtI 
Thtf KlNi) cow fp^^nd 4s u<«0 to 
o^rticvjlir txjru. 1 yo « find X 

tin J erythrocyte 

t I Hit r 1 i ia t i on 
We r ec o mm <»n .1 th^t yoo 
anO th«n comoin^ th%? 
, f I nd comuu t f»r 

f I nJ m^n or 

coMOi'^e setl did sntZ 
Mill find documents aooot CJHf\)TLK HtHOKltS by 
undur BOTH CJHPUTtK or COHPUTtKS» e^c.t (^etl) 
For <4clJitiond'l e><pl .*o»t i ons that «ay be ustrtui 



• ••Tchin^ for, 

• xawpf est 



s t ar t 0 y 
r e su I t v)n t 



searchini on ^in^lii <Kwy) Mords or hot d sfew* 



sets as neeoeui 



t ne 3 co«nand%t 



como in e 
Mod civjthor 
find iHore 
shoN in^ex 



find 
f inrj 



i ndex 
0*? t t er 



getting documents index^^d 
ANO under HtMOR(YfItS» ( se t 2 » 
ton uov^rchinvj typai 
(for combinun*! SMfs of retriwved aocuwents) 
(for searching Oy ajthor na«^9l * 
(for Hi*y.s to t'inl additional docu»«ntsl 
(for broMsing throvjjn the valid 4nde)c terasl » 
(for searching on terms given by SHOi^l INOEXI 
(for ways of getting more relevant docviments) 



eXPLANAriON OF I^UTHOft itAKCHlNG 
in addition to getjitvral s^b|ebt searching* 



specifically on a particular author, 
follo«««d by the author's name in tho 

find author ma^tint r 
Which can oe abbreviated 

faumartinfr 
Hill do a search on authors m tth 
baainning witN \ 

If this givas Vo3 many MaUa 



I o do 
f ormt 



you can request 
this t ype t 
f I ast-n.^m^t 



saarGh 

find, aut;^or 
ini t iall • e .g • « 



the last name 'Martin* and ^ first name 



drops* (e.g.9 you gat documents by diehard 
Hartin when you wanted only. those oy Kooart tviin Hartinit you can be 
moro ^pacific as in the f'ol lowing two axamplest 
f au martin« robert < 

f au mar.tinf r. a ' x 

Notat howavart that diffaraht data basas Vary as to how tnay format 
author namas. Soma usa only initials; 9iome use periods after initials* 
o thars do. not • 

fha oefit strategy j»«y oa to start with a broad 
tome documiints. If /OU than want to rastrict .the 
naaf format that you see in the SHOW potpuf. 



author searjch ana SrIOW 
s^archf use the author 



rrti? o*(|«' last uDiatuat 7^-12-17 



^HOiiil^J iNlEx TtRHS^ 
I 'thoii index <• c oiiii«r^vJ ^ I i s t s t*»ri»s 
t«rn X in thu indvx for th« J«ta t>aSf» ybu 
Th« number ^ocuwents posted under eech 
%^Of ih lex avo jedro 



j| phec^tical I y ne^ir'the 

*r e curr en t I / c onnec t ed t o* 

term is also >iiven» f f^uV^ 



Mill Hhow if AVOGAOp^O Is a u^oper se*rcn,t4rw in yo^r 
lyou can checK spelling). Voo will also ^e« t>ot< wenvt 
doc u wen ts ar posted under t hjt t i»rm, or 41 t ern^te I y , 
like Avoj«dro/s nu«oer or rule or constantf etc. 

For some j'at a o^ses you Mill also oe jiven ah inJicatlon 
tnere are flitlrws for related th<?savirus t«rms. 



data Pu se 



current 
1 f any » 
other rill ated 



^»hr ases 



ot whether 



FINOIN.; /^ITM 
p. ferms that are 
be 



f EKMS 
shOMn 

use J as sVar ch ter ms 
If you are connected 
fro« t>>e displayed list 
\pel Jed-out form ot the 
find e/ 

will do a starch on term 7 in 

find ei-eOteie 
Mill do a io«rch on the first 6 
9 searchos ire combined wlt^ OK t 



FKO.i SHOW INOeX COtlMANO 
oy the SHOW iNOtX commend 
in the jFINO commend, 
to dIALiffGt you cin select 
oy desi<inating tofir code 
t er m 5 • i f hu i f 



mjyt of courset 

por tlculmr terms 

tags instead of the f ul I 



the *>HOH INl>"X Jj^lsplay^ while 



terms .pi u^i t erm Id 
as for synonyms) • 



< r esv t ts of the 



♦♦JO 

FINOING 
Some ways 
Use 'snow 
Use 'or' in 
Get synonymous 



1. 
3. 



rtORE UOCUHtNTS ; ^ 

to find additional dpcdmentsi 

indfx X' commend to find additional search terms 

COHBINt command to '^collect documents unaer synonymous tens 
related terms from 'snow #11' document outoutt 
or from thesaurlt dictionaries, indexes* y</ur he*df etc. 
Break a search p'>rese' Into singl* words 
5m Use other d«ta bases and systems ^ 

6. Use truncated (stemmed) search terms. CONIT searches on 
BEGIN with your search term. Thus, 'flno tomput>r' will 
documents Indexed under 'conputer*, -computers', 'computer pro^r^s 
'computerized', etc. To broaden your search, reduce 
search term. t*^*» 'find compuf wil>>^et aril et thtf docummnts 

PLUS tnoM indexedf^under ' comput e* ^ 'com0u tat 4 on t etc 
from saarch; e.9« 'effects' in "heat effects' 



'computer' search 
arop ')ess important 



al I terms that 
retrieve 

f 

the lenjth ot your 
from the 



t erm ^ 



4 




COiil} L X PL AM! IONS M-^: 



MNOlNi, tiLMtK DOCUhtNIS 
It you h*ve too lany non-rel«v**nt Jacjm^^ntSf your st»irci 
not be spocAtic 9ncf\j^t\ or you way not h3y« includeU an iwu< 



To correct tnist .s«jrch on th« concvipt you r>ava ofifitt^d; 
the new jnd ^irevious search sets usin^ COMdlNE with ANO* 

if 3 major oart of your inarch s t* t Is not relevant and 
isolate the ir t by some *siijrcn» yovJ c «*n exclude th.> non-r 



tr 4tn^co»0 in« 



vl|o u can 
IttvarSt 

p^art oy comolninj tn« ori:jio<*l result AND NOf tne now searqjr) 
f i»nd m^mor 
f i nd c oupu t 

comoi'^e setl and not setZ 
Mill r«»move (at least -iom^ of) 
may also remove %om^i documents 

If you inoH tne retrieval sy s t em ' I aA^ua je t you c in also 
as by datet repor^t numbert ••important terms'* only* ,*tc. t. 

find rn = nasa-C'^-l2952<# 

find antorphous( W> state/ t i 
Some OKOIT or fitOLINE examplesi 

f i nd new ( sa) 

find *l^dc> and •poisoning 



the comput^^r m4yiory Jocument 
on hvjm.in memory Mherii' comput 



HOMever« it 
er s ^are used • 
Mo special searches 

« in OIAlOOS 



••3J 

EXPLANATION OF CCJHOINt COHMANO t 
The CUMBINt comma nd a I • 6ns ^ou to maKe combinati3ns of th 
of^Jocuments you have previously found from searchiov^ your 
conne4;te^ data base; for examplet 

combine se t 2 and set 5 \ 
will yield ) new set . f hat ^conta ins only documents which are 
to both sets ^ ANU 3. ANOin^ is useful in narrowing the %e 
specific topic. Alternatelyt if want to oroaaen yoJt^ searcfr 

combine set Z or set 5 
merge^i' the 2 sets into a single so^t Ke^oin^ all documents 
ORiny is useful in coiiibininj results from synonymous terms. 

f i sul f a te 

f ind sul f i te 

combirie iaf 1 or set 2 
will collect documents about sulfates OR sulfites into one 

combine set 2 ind not set 5 
will make a new set which contains docunents in sot 2 but n 

1^ « 

This pagt last uplatedt 7«-9*a' 



terms may 
tant cooc ept. 



e sets 
urr ent I y 



common 

ir*ch to yoyr 
« use ok; e.gr« 



fft-om eithe?r set. 
The 3 commands! 



jet. Alsof 
1^ in set 5. 



C0NI1 tXPLANAllJUS Ai-^^ 



/ 

Th» srlow comm*n.J jive!* mformatian joout tJ0cu«^n t s^t hat 
VfJ^C^if^ I* ^^<^HL^ Oat<* Dd*«st .aDovit -index 't«rms t :> -iejrth 



oHf etc* 



To have CJNl.T show 
of c1ocviw«nts yovj havt 
shoH set ,3 t i 1 1 « 
Mill c Ju&e the ti t les 

shoM dJi I ci ? 
^ivei ALL -fnf ormation availvibte on 
To hav/fe the SHOW output orinted 
s s/i a M J off <^ 

fill qa^ all info for the first b? 



standard c i t a t i on i nt orma t i on on so«« oT Vhe last ^ct 
found typ« '^now'* You may aKo do wore specific; .i.g.f 
docuw**nts 

of the first documents ot„s.?t J to »€ snomn 



t o ^you 



th« 5tU0Nl) docvi«^r>t of the c'urr«nt sfet 
online and maiMcl^ add *ott'; e.^.t 



jocuncnt s of 



C0HM4N0 
show data 



set J printed offline, 

[•d* is the 3bbreviation tor 'd oc urn en t ( si • t 's' t 'shoW v*nd 'stf). 
For more ietjils on how to jet document, information t yp > i e sho*< d 
Examples yt v)ther information that can oe ootalneJ ^re giy%:n oelowi 

mitf tXPLANAT ION 
Lists data Dv«ses currentljy availaole throutjti CONiT 
Lists ttrms near X in inJex fob cgrrint Jata oase ^ 
Reviews search sets crerated so f jr o^ current data 
Tells wh4t Jc*ta base and system are' o onnec teu 
Gives news from connecteJ system 
For more Jetails on each, typei e s^ow Y 
whttre Y.is Tne command you want explaimed; e.g., 
e show i ndex 



show 
show 
show 
show 



index X 
review 
st atus 
news 



dStt 



» 



f irst 



♦shShin^ oocumemt infokmation . , 

The basrc conmancJ lor showing information on docum en t Sj, found has the formi 
show docu««nts 5-7 ^tt 3 t'i 1 1 e . * . . 

Which causes the titles of JocvJi««nts :.nd 7 of set i to o. outpu . 

(You .ay aporeviat.t 's* for -shoW or -saf and • H' for -docu-entis) 
If. you do not specify th^ SET parameter, you will ^e t the last set 

you have found. ' ^» ♦u. 

If you do not specify the DOCUMENT (dl para««tir, you will get the 
lew documents in the set (usually aoout 5, if there I,. ^. 

You can ^i*t all document inf ormat ion oy suos ti t ut inj all for title , 
If you specify neither 'title' nor 'all', you will ^je t the .standard 
Citation iMormatlo-. wt^ich includes title, author, ^nd locator ^^'^ ' 

Different systems and <lata uases give you somewhat different types of 
information for t»». categories TlTLt. ALL, J^^^.^f.^o. . . 

especially true of p ther- than-b ii> 1 1 ogr aoh i u data oa->es I i he FOUNDATION 
GRANTS or C0N&rteSSI3NAL KtCORO. » . , ♦ . 

Jo sa^e money and online time you «a/ ^ant to ha-e your SHOW ou put printed 
offline (NUT at tereinal) and maile<l. To Jo this aid the word oft', e.vj.. 

Show off al I s5 dl-37 . . v . , 

peans print offline all Inf^ on the first j7 documents ih s.«t 3. 

This oai^e last upJatedi 77*1J-1/ 
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Ca^IT t X PL ANA f IONS J^^-Jl 



\ 

i 



SH0W1N^, StTS,l.>l r^EVlEW |. 
The commjnd 'shoH revl^jN* lists vill the sets you have found in 
sejrchiX; tne curr^^Jly conn*#ct»*u Jvita oasw with ,i short description 
pt each s«tt ^in^if for some' sy st emst the number of Jocum^^nts in »?ach set, 



♦»3b 

:;mOwinu^ Nt WS - . » . 

The comminj • shOH news* wiM give you the neNs trofi the *yst em 

you 4*re currently connected to. Thii is in cjdclitlon to fhe short 

bro<idc.<st ni^s you. nay nave gotten nhen you first connected to tne system* 

No^s fo'* the rtPOuINE systems is only dVoildble o/ pitnlnj a special 

nim news system py typln^jl oicK nim news \ 



SEND COrtHAND 
a System in i ts 



OMn I an .^ua^a».« you way use the 



»»37 

lXPLANAT ION OF THE 
To senJ a command to 
SENO command. t»j«f 

send . f i I e " 
iail connect you to DIALOG file 12 WIlVlOUT jroot>ln>j sets 

send t^/ 1/ 1-20 \ 
Hill .giv4 )ust format i output (doc 

send "print indented tit an | 
will, give title and doc ^n^moer info (or MtOLiNE 
welcome your c ommer\t\;r on CONIT. To maKe a 
•5end comment* command, Fw*" exp I anat ion tyoet e 



lust found • 
ftumoers) f or DIALOG set Z 



and ORUiT/ 
comment use 

comment. 



the 



9et 

CONIT 



your commen ts on 
to ^^Id us in our 



COKHtNTIN 
. «4 1 14 I d t i »te 
you flpd in usi 

To make a commfftt use tt^t *comm«nt* command 
. comment tf»e common l^>jua(|e feature is 
comment Jqrknt or type Is a better namd 

page last upJatedt TZ-^li'l? 




inks 



tne ^lood and Dad fx^atures 
study of informatian systems* 
for eacn line of comment; e< 
yooo Out snoM is a dad name 
than shoH 

f . ■ ■ . 



4* t 



V." V 



if 
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APPENDIX B 

EXAMPLE CON IT SESSION 
> 

As explained in Section 2.2, we present in the Appendix .excerpts 
from the online se'ssion for an experimental user (EU3) as an example of 
typical intferaotions that occur between users and the CON IT system. 

During the session, the user issued 90 commands. The^ are labeled 
Ul, U2, U90. The CONIT responses are labeled CI, C2^ C90: Two ^ 

of the commands, U5 and U6, were not attually issued by EU3 but have been 
■ added for completeness as explained in Section 2.2. , Additional explanatory 
information on jthis session is found in Section 2.3.4 and Appendix D, 

The dialog has been retyped in a :^orma't close to that which actually 
appeared on the original typescript. For a comparison, see Fig. 3 in 
Section 2; Fig. . S is a photoreproduced excerpt of the typescript from 
command, U19 to U26. In this Appendix, material left out is indicated by 
ellipses (...) or by a note indicating where the -full CONIT explanatory 
message can be found in Appendix A. . 



f 
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Ul : START . ^ ' . ^ ' ^ ' 

CL: ■++++CONIT; ■ ' 

, WELCOME tO-CONIT. IF YOU MAKE A TYPING MISTAKE, STRIKfi IHE BREAK KEY, 
WAIT FOR THE USER CUE (**^*USER : : ) , AND THEN RETYPE THE LINE. 
FOR MORE HELP ON HOW TO USE CONIT, TYPE 
HELP - 

FOLLOWED BY A CARRIAGE RETURN; OTHERWISE, YOU MAY NOW TYPE ANY CONIT 
COMMAND, ■ - 

09^6.6 , 

*****USER: : ' * 

U2: HELP ' . . 

Q2: ++++CONIT: 

- THE NAME OF THE BASIC HELP COMMAND IS EXPLAIN . IN USING IT YOU 
' , MAY TYPE EITHER THE COMPLETE WORD OR ITS ABBREVIATED FORM E . THE 
•"command . NAME MUST ALWAYS BE FOLLOWED BY THE ITEM YOU WANT EXPLAINED 
. ' AND BY A CARRIAGE RETURN. FOR EXAMPLE, IF YOU TYPE 
EXPLAIN COMMANDS - 
^ OR JUST , ^ V, . 

B CONWANDS ' 

CONIT WILL LIST ALL THE BASIC COMMANDS YOU CAN USE AND A SHORT- 
EXPLANATION' OF THEM. THE INEXPERT USER SHOULD. NOW TYPE: E COMMANDS 

0.957,6 ^ , 

.*****USER: : 

U3: . E COMMANDS I 

C3: [See EXPLANATION 9 from Appendix A] - 
.U4: E- PICK . « 

C4: [See EXPLANATION 20 from Appendix A] . . 

US: SHOW DATA „ " 

tS: [See EXPLANATION 10 from Appendix A] . 

U6: SHOW DATA 'S ' / \ ' ^ 

C6: [See EXPLANATION 15 from Append i-x A] 

'U7: PICK Sr ^ \ 



. \ 



fc7: +++-fCONIT: , U 



IN ORDER TQ CONNECT TO THE DATABASE YOU PICKED, SOCSCI (NUMBER 51), . 
CONIT WILL CONNECT ^TO THE DIALOG SYSTEM. \ 
Td MAKE THIS CONNECTION CONIT WILL FIRST PHONE THE TYMNET NETWORK. 
THE WHOLE CONNECTION PROCESS WILL TAKE ABOUT A MINUTE OR TWO. ' 
4^ ' PLEASE WAIT FOR THE USER- CUE t*****USER: :) BEFORE MAYING A NEW COMMANp. 

']f^i^}::.yr --^-''. ^v^NOTE THA^ THE BREAK KEY tAN BE USED AT ANY TIME INTERRUPT . CON IT 
1^% :- AS^ELL AS TO CANCEL' A LINE : WHEN, YOU STRIKE THfi BRE/^K KEY CONIT'WILL 

^S^';;"!-^^^^^ MAY (SrVE A NEW COMMAND aWiC YOU GET 
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. C7: (cont'd) 

PHONK CONNECTION MADH TO TYMNET. 
TYMNET RESPONDING. 
LOGGING ONTO DIALOG. 

you ARE NOW CONNECTED TO THE LOCKHEED DIALOG RETRIEVAL SYSTF.M-- 
YOU ARE NOW CONNECTED TO THE SOCSCI (NUMBER 51) DATA BASE. 
'FOR EXPLANATION OF FINDING d6cUMENTS TYPE: E FIND 

1006.1 ■ - 

*****USER:: ' 

U8: ^E, FIND * ■ \ 

C8: [See EXPLANATION 26 from Appendix A] , 

U9: E COMBINE . 

C9: [See EXPLANATION 32 from Appendix A] 

UIO: E FIND MORE 

CIO: [See EXPLANATION 30 from\ Appendix A] 

I 

Ull: E FIND INDEX ^ ' . 

Cll: [See EXPLANATION 29 from Appendix A] 

U12: E SHOW INDEX 

C12: (See EXPLANATION 28 from Appendix A] ' ^ ' ■ ■ 

U13: E FIND BETTER - 

C13: [See EXPLANATION 31 from Appendix A] : 

' ' '* ' 

U14: E FIND AUTHOR ^ 

C14: [See EXPLANATION 27 from Appen^x A] 

CIS:* PHONE LINE DETACHED- 

THE PHONE CONNECTION AND RETRIEVAL SYSTEM HAVE BEEN DISCqNNECTED 
FOR OTHER SYSTEMS THAT MAY HAVE THE iNFORMAtlON YOU WANT, TYPE: E 




U16: PICK 5l 



U17: SHOW INDEX MODKL 



C17: ++++CONIT: ^ 

REF INDEX-TERM TYPE ITEMS RT 

El MODALITY- .149 

E2 -MODALS---r----'-- — — \5 - 

P3 MODDERMAN 1 

■E4 . MODE- 267 ^ 

E5 MODEC/VTE - 1 

E6 -MODEL 6355 

E7 MODELE-- 1 

E8 MODELED 19 

E9' MODELER---- --— 1 

ElO MODELES 2 

' Ell ■ MODELING 636 

E12 MODELIZATION 4 

E13 MODELL 7 

E14 MODELLANSTALT 1 . • " 

E15 MODELLE---- 1 < 

E16 MODELLER 2 ' , 

E17 MODELLING 82 

E18 MODELLKINDERGARTEN 1 

E19 MODELO 1 

E20 M0DELR--1 — i 

TO SEE MORE TERMS TYPE : SHOW MORE 

TO SELECT ANY OF THESE TERMS FOR SEARCHING TYPE: E FIND INDEX 

1026.0 ^ J 

****^USER:: " V> 

U18: SHOW INDEX WORLD 

CIS: •*+++CONIT: 

REF INDEX-TERM TYPE ITEMS RT 

Ei WORKWEEK--- 14 , 

E2 • WORKWEEKS 2 

E3 WORKWIFE- — — 1 

E4 WORKYARDS — 1 

E5 WORLABY 1 

E6 -WORLD-- — - 6520 

. E7 WORLDLINESS-— — 1 

E8 WORLDLY-— - 6 * 
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U19: , FIND MODEL 

C19: ++++CONIT: 

NO RESPONSK RECHIVED FROM IAST'TRANSMISSION. 

SHOyLD CON IT WALT FOR A RESPONSl: ANY LONGER? (ANSWER YES OR NO) 

RETRIEVAL SYSTEMS MAY BE SLOW IN RESPONDING AT TIMES. YOU SHOULD 
ANSWER YES TO THE ABOVE QUESTION ABOUT 3 SUCCESSIVE TIMES (UNTIL ABOIH' 
A MINUTp HAS ELAPSED) AFTER WHICH TIME YOU MAY ASSUME EITHER FHE. 
RETRIEVAL SYSTEM OR CONIT OR THE NETWO^ie CONNECTION BETWEEN HHE TWO 
IS NOT teKING PROPERLY » AT lllAT POINT YOU SHOULD ANS_WER NO TO I'HE 
QUESTION. THEN. TRY YOUR REQUEST AGAIN. IF YOU STILL GET NO RESPONSE, 
DISCONNECT FROM TOE RETRIEVAL SYSTEM (TYPE DIS) AND TRY TO PICK IT 
AGAIN (POSSIBLY USING A DIFBERENT NETWORK) OR PICK A DIFFERENT SYSTEM. 
*****ySER:: 

U20: ■ YES , ' 

C20: NO RESPONSE RECEIVED FROM LAST TRANSMISSION.*** 

SHOULD CONIT WAIT FOR A RESPONSE ANY LONGER? (/VNSWER YES OR NO) 
*****USER: : 

U21: YES 

C21: FROM HOST: 

YOUR SEARCH RESULTS IN SET 1 WHICH CONTAINS 10415 DOCUMENTS. 
TO SEE REFERENCES TO TOE FIRST FEW DOCUMENTS TYPE: SHOW 

1034.5 

*****USER:: j " 

« 

U22: FIND WORLD 

C22: ++++CONIT: 

NO RESPONSE RECEIVED FROM LAST TRANSMISSION. 

SHOULD CONIT WAIT FOR A RESPONSE ANY LONGER? (ANSWER YES OR NO) 
*****USER: : 

I 

U23: YES 

f 

C23: NO RESPONSE RECEIVED FROM LAST TRANSMISSION. 

SHOULD CONIT WAIT FOR A RESPONSE ANY LONGER? (ANSWER YES OR NO) 
*****USER:: 



U24: YES 



024: FROM HOST: 

. . . YOUK SEARCH RESULTED IN SET 2 WHICH CONTAINS 6904 DOCUMENTS. 

TO SEE REFERENCES TO TOE FIRST FEW DOCUMENTS TYPE: SHOW 
FOR AN EXPLANATION OF HOW TO COMBINE SETS TYPE: E COMBINE 



\ 
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C25: ++++CQNIT: 

Yam St^ARCH RliSULTlil) JiN SHT S WHICH CONTAINS IKS IKXlUMIiNTS. 
TO.SIJli RKFERENCDS TO THli FIRST FHW DOCUMHNTS TYPi:: SHOW 
' TO RCVIi:W YOUR SKARCH SHTS TYPR; SHOW RllVliiW 

1037.9 

*****USHR:: 

U26: SHOW 



• C26: ++++CONIT: 

3/3/1 
. 785089 

HOUSING RKQUIREMIiNTS AND NATIONAL RHSOURCHS - IMPLICATIONS OF UN 
WORLD MODEL (^N) 

LAKSHMANAN TR; CHATrRR^IEE L; ROY.P 

JOHNS HO^^KINS UNIV, CTR METROPOLITAN PLANNING <i 

RES/BALTIMORE/MD/21218; JOHNS HOPKINS UNIV, DEPf. GEOG ENVIRONM 
ENGN/BALTIMORE//Ml)/21218; GALLAUDET COLL/WASHINGTON//DC/20002 
" HABITAT, U2, N3-4^ P277-289, 1977 ' 

3/3/2 . 
783666 " ^ . 

ECONOMIC-MODELS OF WORIi) AGRICULTURAL COMMODITY MARKETS - COCOA, 
COFFEE, TEA, WOOL, CaiTON, SUGAR, WHEAT, RI€E - ADAMS, FG AND 
BEHRMAN, JR (EN) , . .. 

- JONES GT 

, . JOURNAL OF AGRldULTliRAL ECONOMICS; U28, N2, P326-320, 1977 • . 

/ 3/3/3 ' ^ - \ 

777057 ' . . ; 

BBHAVIORAI-MODEL FOR HEALTH PROMOTION AND DISEASE PREVENTION .(EN) 
NG LKY; MANDERSCHEID RW; DAVID DL 

NIMy^WASHINGTON//DC/20032; WORLD MAN PUND/WASHINGTON//DC/ 
JOURNAL, OF ALTERED STATIBS OF CONSCIOUSNESS, U3, Nl, P95-95, 1977 ■ 

3/3/4 ' ' . 

. ' 772792 

BARILOCHE MODEL - 3RD WORLD DISASTER STUDY PREDICTS (EN) . 
JAHNKE M ^ ' 

' ENVIRONMENTAL POLICY AND LAW, U3, N2, P74-75, 1977 ' . ' , 

• •" ^ -3/3/5 ' ■ * ' • ^ 

. 759229 ' ' , ' . ' «i 

HUMAN IZATION OF LABOR WORLD - NECESSARY CONDITIONS,- (iONCEPTS AND 
V ' MODELS GERMAN - PREISS, C (GE) 

; , TRA0TWEINKAIM3 G : ' V 

f^- v . ! 'GEWBRKSCHAFTLICHB MONATSHEFTE, U28, N7, P469'-469, 1977 

.0^;.^.:.^ z.^ ■ •.; -vv -cfr ■■1 Or:' ■ ■ ■ '^^ • - ■ ' 

■'^F RI O' X •-• -''v.' ■ ■ , . . ■ .■ .-, ■■■■ 



YOU MAY NOW Sllli O'niliR INI-ORMATION ON YOUR SliAUCll IU;,Sl!l,TS; TYIM'.: 1-. SHOW 
I-OR i:XPI,ANATlON Ol- OlUliR ACTIONS YOU MAY NOW DO'TYPfi: ii COMMANDS 

1044.1 
*****USi;Rf': 

U29: SHOW SHT '5 ALl, 1)1-113 OFF 

C29: ++++CONIT: 

PRlNTIil) 3/5/1-113 ' 
... YOU MAY NOW SKI- anil'R INFORMATION ON YOUR SFIARGI RFSULTS; TYFF: F SHOW 
FOR FXPhANATION OF aiHF,R ACTIONS YOl) MAY NOW LH) TYPF: F CONWANDS . 

1046.3 
*****USFR:: 

U30 5 SHOW INDFX' DYNAMIC 



U31; FIND DYNAMIC 

C31: ++++CONIT: ' % 

NO RFSPONSF RFCFIVF-D FROM IJ\ST TRANSMISSION. 

SHOULD CON IT WAIT FOR A RFSPONSF ANY LONGFR? (ANSWF!r-¥t-S OR NO) 
*****USF.R:: 

ii 

U32 : YES 

C32: FROM HOST: ^ 

YOUR SFARCH RUSULTF.D IN SHT 4, WHICH CONTINS 2135 DOCUMENTS. 
TO SFE REFFRHNCFS TO T\\^ FIRST FEW DOCUMENTS TYPE: SHOW 

1049.1 ^..-^ 
*****USER: : 

U33: COMBINES^ [User hits BREAK key after 'S'] 
C33: ACTION/ MiSSAGES ABORTED. CONTINUE. » 



1^ 1049.6 

^ *****USER: : 

U34 : COMBINE SET 2 AND SE1> 4 



U53: ^ COMBINE SET 10 AND NOT SET 3 



U56: PICK 73 



C5j6": ++++CONIT: 

•LOGGED OFF DIALOd. COSTS FOR THE USE OF LAST !)A! A {iAsSF : 
$64.12 0.91^ HRS FILE? 5 DESCRIPTORS' ' 

$7.33 -TYMNET ' , ' 

$12.10 121 PRINTS 
' • $83.55 ESTIMATED TOTAL COST . . ^ 

TIME OF LOGOFF^ 8: 18: 24 ' - ■ ' ' '' ' 

LOGGING ONTO. SDC/ORBIT. ' 

FULL RESI^ON§E YET RECEIVED FROM itETRIEVAL SYSTEM.- . ^ 

SHOtJLt) CONIT WAIT FOR A RESPONSE ANY LONGER? (ANSWER YtiS OR NO) 
*****USER: : 

USJ: YES ■ • 

C57: NEWS FROM SDC/ORBIT: 

•SAE DATA BASE NOW OPEI^ATIONAL . SEE NEWS RE FREE INTRODUCTORY TIME 

***** 

YOU ARE NOW CONNECTED TO THE SDC/ORBIT RETRIEVAL SYSTEM--' 
YOU AHE NOW CONNECTED TO THE SSIE (NUMBER DATA BASE. 

• . FOR EXPLANATION OF FINDING DOCUMENTS TYPE: E FIND 

1117.9 ' 
*****USER: : 



U78: .;5H0W SET SHOW SET 5 Dl-5 

C78: ++++CONIT: ^ > 

A number shduld appear where you typed "show set 5 cJi-S" 

For an explanation of how to use the SHOW Gommand type: e show' 

1136,4 ^ 
*****USER: : 

U79: SHOW SET 5 Dl-5 . . - 

\. / ■.. * / •■. \ : 

U84: S REVIEW , 

C84: ++++CONIT: 

set 1: ALL MODEL: (43383) 
set 2: ALL jWORLD: ^ (46581) 
■ , ^et 1'.,,^ 1 AND 2 .(606) 

. ' S set 4}: 'ALL DYNAMIC: (11555) 
. . ' set l\ 2 AND 3 (133) . , 
set 'S: 5 AND NOT (79)^^ 
S . set . 7: "P (4853) \ . . ' 

; set .8: ALL SOCIAL: (37749) 

'MIX ' ■ ■ iiu.i . :■ ' :■ 



-104- 



U85: COMBINH Sl-T 8 AND SHT 3 . 

C85: +«^++CONIT: 

Your search resulted in j^et 9 which contains 198 docuinen 
. To see references to the first few dpcumeg^ts tyj^e: show 



-AN - GY 68729 1 

TitLE: MODELING REGIONAL GROWTH FOR EFFECTIVE AGRICULTURAL PLANNING 

AND ITS IMPLEMENTATION 
IN - LIANG, T ; VIETH G ■ 

OS - UNIV. OF HAWAII, AGRICULTURAL EXPERIMENT STA., AGRICULTURAL 
ENGINEERING, GILMORE HALL, ROOM 102, HONOLULU, HAWAII, 96822 
PP - 10/76 TO 9/77 

FY 77 ^ • ■ 

SPO - Hawaii State Government, Honolulu, Hawaii 

CN - 0068729; HAW00523-S 



1144.6 
*****USER: :• 



U86:- SHOW 



C86: ++++CONIT: 



-1- 



V 
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AN 



GY 43458 




087 : [User hits BREAK key while output is coining from remote ;host] 



G87: Act ipn/mes sages aborted; you may continue when you get the USER cue. 



,1145.8 
*****USER: : 



US^: SHOW ALL SET 9 Dl-198 OFF 



ior>- 



C89: ++++C0N1T; . 

pending mail address 

YOUR OFF- LINK PRINT NUMBER IS P0084524. 

OFF-LINE PRINT JgMPLRTllD. 1 , 

'A 

You may now see other information on your search results; type: 
For explanation of other actions you may now do type; e command 

A V . . . ■ - , 

1147.0 ■ ' 

-*****USKR:: 

U90: STOP 

♦ 

C90: . ++++CONIT: 

End of SOC/ORBIT session. Time statistics: 

TERMINAL SESSION FINISHED 12/20/77 ,8;45 A.M. (PACIFIC TIME) 
ELAPSED TIME ON'SSIE: 0.50 HRS. 
TOTAL ELAPSED TIME: 0.51 HR6. • 

Logged -off SOC/ORBIT. , " 

y The phone will be hung up. 

Phone hung up. x.. 

Phone line detached r 

ITiank you for using CON IT. 

12/20/77 1147.8 est Tue 

You may hang up your connection now'. 
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APPENDIX C 
OFFLTNR INSTRUCTION MATHRIAL 

f 

The materials reproduced in this Appendix were presented to the 
experimental users as explained in Section 3. The briefing was read to 
each'^U The pother materials were given in the form of. a printed handout 
to the three EU' s from Group A. 

List of Materials 



/ 

1. Briefing 

2. How to Develop a Search Strategy (two pages) 

3. Attachment 1: Work Sheet for Search Terms and Databases (one page, ' 

as filled out by EUl) ^ 

4. Attachment 2: Professional Fields and Corresponding Databases (seven 

pages) . • ' 

5. Attachment 3: Database Descriptions (one page of 14 in Attachment 3) 

6. Attachment 4: Alphabetical li§t of Databases, by CON IT Name and 

Number (one page) 



n 



11 
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PRR-SOSSTON BRTFFING FOR FU'S 

/ 

We are attempting to design a^ computer interface that willl enable end 
- users to do their own searching in machine- stored bibliographic databases 
We want your help in evaluating an experim^mial interface called CONIT 
that allows you to coranunicate with three different Infojhnation Storage 
and Retrieval Systems located in various parts of the coantry. 

Our purpose in this experiment is to deterjpine whether^ tjr not CONIT has 
all the features that are needed to enable you to do yout own searching. 
We also want to know about the undesirable or inconvenient features of 
the system as well as the good ones. 

Although one or more experimenters will be on hand to help you out of 
difficulties, we would like to see if you can master the system by your- 
self and extract useful information from it. Please ask for help only *■ 
if you feel hopelessly hun^, up. 

We shall also app- oc i t ^ your staying with us for a debriefing session ^ 
aftfer^you finish o.i l iving. Ho^ much time do you have today? 



To assist us m i!(ri)reting results of the experiment, kindly let us know 
ibout any ope: 1.3! ( t actions and impressions you have as you prog;ress. . You 
can express them 01; lly, or note themXon a pqd or on the printout paper. 



Printouts of references can be obtained online or offline. We request 
that you limit the offline references to-^ total' of 200. These will.be 
furnished to you free of charge when we get them, in about a week. We 
, also plan to do out own search on your problem. Any new documents that, 
seem relevant will be sent to you. - * 

Please do n^ be afraid that you may damage the system; it is essentially 
damage -proof. 

We lire required by law to ask your permission to use the rfes'ults of this 
/experiment in our report. We shall identify participants by their EU 
numbers only. May we hilve your permission? You are, of course, free to 
discontinue the experiment at any time. \ , - 

(Thp following was read to EU's in Group A only.] • \ 

Here are instructions for developing a search strategy. When you think you , 
have a good set of search terms and databases in which to search, please 
begin working at the c^sole. Kindly leave the list of search terms you 
have compiled when yow leave. 
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HOW TO nt?VKLOP A SF.ARCM STRATLGY 

Before using CONIT, you should plan a search strategy. Here are some 
suggestions: 

1 . . Compile a List of Search Terms 

» 

Write dovm a list of words that describe the topic on which you are seek- 
ing information; a work sheet is attached for yo\ir convenience. Your list 
should include broad, general terms {such as 'the' professional dlscipline(s) 
or subdisfcipline(s) to which youi* pi^'oK^em is related], less general tentst 
and narrow, specific terms. Brei^k your search topic into concepts and list |, 
words, that characterize the concepts. Try also to think of synbny»s for the 
words ypu write down. This list should help you get started, but it nay turn 
out that additional terms will show up as you proceed, 

We want to emp][iasize strongly the importance of listinamany different 
words and phrases that describe your problem. j 

2. Selection of Systems and Data Bases * • 

V^Enter on your work sheet the data bases that are likely to <^ontain docu- 
ments pertaining to your search terms. An alphabetized list of professional 
fields and data bases that cover the fields is attached. Also appended is a 
detailed description of each data hasd. Try to matdfi data bases to your search 
terms. Bear in mind that, foi: completeness, you may want to search more than 
pne data base; the computer will explain how to change data bases'. 

3. Hints on Searching * ^ ' 
Getting Started • x 

Start with the search terra you think will most likely yield useful docur- 
raents. In general, it is preferable to use single-word terms that indicate 
specifically the kejf concepts in your topic. You can ask'the computer to 
explain how to combine results from your single-word searches. Instructions 
on how to »ake your request are available in (Jp>^T. \ 

Documents are often indexed, in part, from a carefully controlled classifi- 
cation scheme an^ a controlled vocabulary. To get started, therefore, you «ay 
have to .guess what som of these terms are. Sometimes your search term Will 
yield no documents only because its form differs slightly from the allowable 
one. / For example, a search on •automatization* may be negative because the 
allowable term is ^automation* . To determine if your term Is close to an 
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Search Strategy 

acceptable tepn, you may ask for a display of IndeJt terms in the data base 
that are alphabetically near your term. Your tern will then be placed in ' 
alphabetical relationship to allowable terms^ suclv^s i|t the example, just 
cited, autc^ted, automation (automatization], automaton The coiwand 
for accomplishing this is: show index followed by the search term you 
want placed in alphabetical context. 

Once a few documents are found, you may discover additional g<^ search 
terms by looking at all the information available for each document. The 
command is: show all. Details of how to use this command will be availa- 
ble when you get online. ~ , 

No Documents Found , ^ , \ 

• If no -documents are found, tty searching on synonymous terms, or try 
broader terms, or look for valid terms with the show index command. 

Too Many Documents Found 

If too many documents are found, most of which appear irrelevant, you 
may reduce the number of irrelevant documents iix these ways: 

a) By searching ort narrower, more specific terras 

b) By followiVig a search that yields many irrelevant documents with 
one or morp searches on narrow specific terms, and then combining 

I the new sets with the first set. You can ask the computer to give 
an explanation of how to do this. * 
^ c) By Sfjarching on two- word and three-word phrases^. . . 

A final hint: Be flexible as your searching progresses. There are no hard 
and fast ^ealtch- strategy rules. Be prepared to alter your searth plan as . 
you progress, . / 

♦ When you feel you have an initial search plan, begin your searching at' 
the CON^T terminal. Stepf-by-step instructions will %t given to you by CX)NIT, 

.-'>". " ■ ' * " ■ ' 
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PO&LIC 
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fRDFtSSIONAt FIELDS WU CORRESPOSOING DATA BASES 



Accounting 

(C) ACCOUNTA.VT, 62b 

tG) Bl«, 61; >»ANAffl, 6?a 
Acoustics 

(G) niYS, 12; fC, 27; CNGIM. 21 

See alsoT Phyiicf 
AeronMiticf 

(O)tNOIN. 21; >ms, 71; SCI, 11 
Agi icultiirW 

(C). AGRICOM; «lb; AGRIRES, 41r; AGRIUS, 4U 
(S) POOO, 41c 



Audlo-visiMl M.it«rrifAi 

(S) FDAV, 56c; MiOAV, 45b 

B 



' V » 



BAjikinf 

(G) KANAGc/6l; FED, S4t; BUS, 5 
Sm ulio: Econonici 
ilochniitry 

(G) MEOLI\E, 45; a«E'!77, Ua; CHEK72, 15b; 

CHeM70, 15c; QIF.HIN. 151; BI0SI5, 42; SCull; 
KTIS, 71 



See alio: Businass. Enflneering, General, Nutrition, .,^.1 , ^ i ^ . 

- $«• also: >Mlcal Sclent©!, CheMlitry, Lif« 

Scienca 

Air Pollution 

(C) AIRPOLLUr, 55b 



(G) POLLirr, 5Sa; NTIS, 71; ENVIR, 34 
Sao alio: Environ»«ntAl Sclanca 
Alualnua 

(C) AL^N. 24b 

(G) METALS. 24a; ENGIH, 21 ^ 
Aniul Scianc« 

Sat; Lifa Sciancas^ Africultura 

Ant^iropolofx 

(G) SOCSCI, 51; A^3IIST, S7b; SOC, 52; BIOSIS, 42 
. " Saa also: Social Scioncai 
Aquatic Sciancai 

(C) OCtAN^V^Sa; AQAU, 33b 
(B) BIOSIS , 
Archaeolpi/ 
(G) sdbci. 5ll ^ 
Sat aifto: Hiitlxy 
*Art 

.(C) ART, 5^ 
Artif^icial Intalliitafica 

(G) EC. 22; NTIS^yi; CNCiN* 21; te/23.|M|BCI, 51 

A^tnnomf 

{Q) hits'. 12; SCI, 11; KTIS, 71 
(f) MBTCOH, 52 ^ 
Astrophx^ici ' * 
' (C) HfTS^ 12; SCI, 11; KT», 71 
(S) MET«», 32 
. AtMsptwric Scianc«s 

(C) KTIS, 71; MrrtiOt, 52 ' 
> ($) AIRPOtUir, SSb 

Sat aUo: Earth and Space Sciancai 



Sciancai 
Bioan'f inaaring 

(C) >tDL!|NE, 45; ENG1N, 21; KTIS, 71; BIOSIS, 42; 
EC, 22 

VSaa also: Lifa Sciancai ^ 
Blolofx 

(C) BIOSIS, 42 

Saa also: Ufa Sciancr*, Dictionaries (p, 7) 
fiOMtrlca 

(C) BIOSIS, 42 

» 

Saa also: Lifa'Sciancas 

Biophysics ' 

(C) BIOSIS, 42; PHYS, 12 ' C 

Saa also: Life Sciences 
Botany 

(G) BIOSIS, 42; AGRIUS, 4U; AGKICOl^ 41b^, AGRIlteS, 41r 
See also: Lifa Sclencea 
Business ^ , \^ 

(C) BUS 41 ^ 

* W ACCOUNTANT, 62b; aiEMJ«WS. ><ta; OmiGNEWS. 66b; 
MARKETAB, 63«; HAMCfiTIft, 6ft; MUUCETMOir, 63ii; 
OILNEfcS, J7w; STAtUS, 64b; STATUSTM, 64c; / 
STATREG, 64d; STATFRN, 64f; STATFRKTWe, 64g; 
STATPLWT. 64a 
$— Blior EconoAica, MMiagMant, Spaclal .Data Baaas 

(p. 7>. i ^ y 



C ■ data bases that arc fntrfil to tiM diacipliM 

fi • data basea that cover the disciplina and 
etkara, too 

S » data bases for a specific aubdisclpline 
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Profe»sion.il Fjiclds and Corresp. Oata P.K.ct 

C 

Cancer 

(C) CANCF.R» 46; CANCERRE5, 46r 

S«« Also: Xtfdical Sclenco 
Cardiology 

S%«: Medical Sciences 
Chealttrr - 

(C) CHi-H77, 15a; eMBM72, CHEM70, I5cs 

CHEHIN» 131; 

(G) SCI, 11; MBULlNfi, 43; NTIS. 71 

(S) CHE^^*EXSi 66a; PATCHEM77, 2^a; PATCHEM50»26b; 
PATDIE^JCA 26c; PAPFR, 26p; TOX» 44; 
TPXEFF, 44d 

Se« also: Dictionaries (p. 7) » ^ledlcal 
Sciences » Patents 
Chemical Engineering 

(G) EKGIN. 21 I 

Sec also: CheiAstry 
Child Abuse 

(C) CMILDAB, 52b 

See also: Education 
Clioatolog/ ' 

(O HETIOR. 32; ENVIR, 34 

See also: Earth I Space ences 
Clinical Sciences - 

'See: >todic^l Sciences 
CoMunications 

See: Engineering or Library and Information Set- 
vices» Linguistics 
Coi^Mtfttional Linguistics 

(C) EE. n; SOCSCI, 51; KTIS. 71; LANG. S$a 
Coi^ter Science 

(C) EE, 22 

, See also: EUctricAl Engineering; Engineering 
Construction 

(G) EHCIN, 2i; MTIS, 71; ME. 23 

See also: Business 
Contracts 

See: Contracts and Grants, p. 7 
Counseling 

(C) EO. 56>; PSYCH, S5- 

0 

See also: Education. Psychology 
Current Events " 
(O PEWNOW, 54n: FED, S4a 

(5 CRECOHO, S4b; CHEWEKS, 64a; OWGNEWS, 66b; 

OILKBKS» 37w 
$ee also: Political Science 



0 

Do»ograp]iJ^ 

(G) SOCSCI. SI; SOC. S2; STATUS, 64b; STATIISTIME, 64c 
Denaatology 

See: Nodical Sciences 
Dissertations 

(C) THESCS. 74 
Drugs 

See : Phamacology 

E 

Earth and Space Sciences 
(C) METEOR. 52 
(G) SCI. 11; NTIS. 71 

(S) GEOREF. 31; OCEAN 33a; AQUA. S3b; EMVIt. 34 
See also; Clinatology. Environnental Science. ^ 
Geocheiiistry. Geology. Oceanography, Soil Science, 
Space Science 
EconoMlcs 

(C) BUS. 61; MAHAGE. 62 

(G) PUBLIC, 54c; SOCSCl, 51; HTIS, 71. HIST. "57»; 

FED. 54a; ANMIST. 57b 
See also: Business 
Education 

(C) ED. S6a; VOTECH. 56d 

(G) SOCSCl. 51; PSYCH, 5$; LAVC, SS 

(5) EOEXCHILO, 56b; EOAV. S6c 
See also: Patents 

Electrical Engineering 

(C) EE, 22 ^ 

(G) BNGIN,' 21; SCI, 11; NTIS, 71; Mi« 2S 

See also: Engineering.- Business, Patents 

Electronics 
See: Electrical Engineering 

Energy 

(C) ENERGY. 36; OIL, 57a; OIUJEbS, STir 

(6) ENGIN. 21; >C, 23; ENVIR. ^ 
See also; Physical Sciences 

Engineering 
(() ENGIN, 21 
(G) SCI. 11; NTIS, 71 
V -(S) BE. 22; ME, 23; METALS, 24a; ALUN, 24b 
See also: Bioonglneering, Physics. Chenlstry, 
Business 



C ■ data bases that ar^ central to thf ilitcipline 
o-data bases that cover the discipline and others, too 
S ■ data base;i for a specific 5Ubdiscipliiit 



; 



Professional Fl«li!s % Corr«sp. Data Batet 



Fntonolofy 

(t) BIOSIS, 42 

Sc« alio; Lift Sciencot 
tnvironmentftl Sclonc* 

(C) CNVIR, 54 

(G) SCI, 11; NTfS, 71; MAAKETAB, 65*; ENGIN, 21; 

tNERCY. 5b; AGRIUS, 41a; 25; 

AGRICOM, 41b; 
(S) BlOSfS. 42; CEORKF, 51; OCEAN, 55a'' 

POLLUT, 55a; AIRPOLLin", 5Sb; AQUA, 55b; 

METEOR, 52 
Epilepsy 

(C) EPIL, 45 

See also: ^todical Scitnces 

Ethnography an4 Ethnology 

(G) SOCSCI^I k 

(S) HIST, 57a; AMIIST, 57b 
Ethology 

(G) PSYCH, 55 . 

See also:x^ Lift Sciences 



Fish 

(G) OCEAN, 55a; AQUA, 55b; BIOSIS, 42 
See also: Agriculturt, Lift Sciences 

Food Technology 
. (C) FOOD. 41c 

(G) BIOSIS. 42; ENGIN, 21 

' Stt also: Nutrition ' ^ 

Forostry 

« (G) AGItIUS» 41a; AGRICOH. 41b; AGRIRES, 41t 

Stt also: Earth ind Spoct Scitncts 
Fouiyiat ions , Charitablt - pagt 7 

G 

Gtneral (All Fitldf) 

(C) KTIS. 71; LIBCONE, 72t; LIBCONF. 72f; 
HffiSES, 74 
Gtnttics 

(G) BIOS^S. 42; MEDLINE. 43 
V Set also: Lifa Scitncts 
Gtochoftistr/ 

(C) CEOREF, 51; OIL, 37« 

Sat also: Chosiistry, Enrth | Spact 9citncts, 

Ctoldtx ^ 



Geography 
^) SOCSCI, SI 
Ste also: Social Sciences 
Geology 

(G) GCOREF, 51 

Stt alsoT Farth ^ Space Scienct| 
Gtophyslcs 

(C) GEOREF, 51 

Stt also: F.irth I Space Sciences, Physics 
Gtoscltftces 

Stt: Earth ( Space Sciences, Physical Sciencts 
Grants 

Stt: Contr/icts and Grants, p. 7 



H 



History 

(S) HIST, S7a; iJmST, S7b 
Set also: tht professional fitld about which histor- 
ical infonnation is being sought 
Horticulturt 

Stt: • Agrlculturt 
KuAanitits 

(S) HIST. 57a; AFWIST, 57B; ART. 58; LANG, SSa; 
LIB INFO, 55b 
Hydrology 

(G) METEOR. 52; NTIS. 71 

Set also: Earth % Spact Scitncts 



Inminology 

(G) BIOSIS. 42; IfDLtN^. 45 

Stt aXso: Lifa Scitnc^.s 
Industrial Enginttring 

J[G) ENGrt. 21; ME, 25; EE, 22; NTIS, 71 

Stt also: Busintss 
InfoTMtion Scienca 

(C) LIBINFO, SSb • ^ 

(G) SOCSCI. 51; EE, 22; NTIS, 71; ED, S6a 
Inaacta 

Stt: Ento»olo|y «^ 
ItistrtJMtntation Technology > 

(O ENCIN. 21; EE, 22; ME, 23; NTIS. ^1 1 
Inttmational Affairs , 

(C) rtliLIC. S4c 

Sat also: Political Scitnct 

C*- 4ata bA5es that arc central to tht diaciplint 
G • data bases that cover tTko" uisclplinp I other*, t< 
S • data basc^ fur a specific subdlsciplino 
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l»»ofc5iionnl flclJi 5 Corro^p. Data flo$e» 



K 
L 



Law 

(<») 



FED, SU; FEOWOW, 54n; CRECORO. 54b; 
BUS, 61 

See: Political ScUnc« an<S alto the proftttlonal 
f iftldt about vhich Usal infoiMtion is being 
sought. 

Library and Tnfor|Mtlon Services 
(C) LlBlNF0..5Sb 

(G) SOCSCI. 51; ED. 56a; KTI5, 71; EE, 22; 
AGRIUS. 41a 
V>lfe Sclencei / 
(C) H0515, 42 

(G) AGRIUS, 41a; AGHICOM, 4!b; AGWRES, 41r; 
MEDLINE, 45; SCI, 11; KTIS, 71; SOC, 52 

See also: Nodical Sciences 
Linguistics 

(C) LANG, 5Sa 

(O SOCSCI » 51; ED, S6a 

See also: Conputational Linguistics 
Literature / 

(€) tANG. SSajSOCSCl, 51 



M 



Manage«ettt 

(C) MANAGE, 62; iUS» 61 

See also:- Business, E^oeiics ^ 
^taterials Science 

(C) PHYSICS, 12; RT1S» 71 

(5) AUIM, 24b; ICTALS» 24a; PAPCR, 26p 

See also: Cheaistr/, Engineering 
Mathematics . 

(G) SCI, 11 

See also: .Accounting, Computer Science, Statistics, fmm 

Nu<riea 



MciUcftl Sciftncrs (contlmi«d) 

. Sco aUo* Cancer. Fpl lepiyi'^ Dkct ionarie* (p. 7). 

Nutrition. Phsrmcology. Piytholofy. Toxicology 
Metallurgy 

(C) fCTALS. 24a 

MetaU 

(C) METALS, 24a 

(G) I^NGIN. 21; niYS. 12 
* (S) ALUM, 24b 
Meteorology 

(G) l^ttTEOn 52 

See also: Earth and Space Sciences 
Microbiology 

(Gf glOSlS, 42; MEDLINE, 45 

See also: Life Sciences 
Mining 

(G) ENGIN, 21 

See also: Metals 
Molecular Biology 

(G) BI051S, 42 

See^also: Life Sciences 
Motor Vehicles 

(G) EKGIN, 21; ME. 25; SAC, T 



N 



Natural Resources 

See: Environoental Science 
Naval Technology 

(G) ENGIN, 21; OCEAN, 55a; NTIS, 71; NB. 25 
Neurology , ' 

(5) ePIL, 45 

See also: Medical Sciences 

Noiso Pollution 

(6) POLLVT, 5Sa: ENVlR. 54; KTIS, 71; AGRtUS» 41a 
Seo also: Environskental Science 



Mechanical Engineoring 

(C) MB. 25 ^ 

(5) PATENTS, 25 

S«e alto: Enginoorinf 
Mochanict 

(G) PHYS, 12: mj 23 

$99 9\90i Physics 
Medical $cienceft 

(Q ICDLINE, 45; MEONOM, 45« 

(6) ilOSIS, 42; SCI, 1|; KTIS. 71 
(S) 1«0AV, 45v; MECMON, 4*i 



ear Science and Engineering 

(G) PflVS, 12; tHGiN, 21; EE, 22; NTIS, Tl 
See also: Physics 
Nucleonics 
(G) PHYS, 12 
See also: Physics 



C • data bases that aro central to the diteiplin«. 
G data bases that cover the discipline % others » u 
S *> data lUses for a specific sUbdiscipliM > 



Prv)fc»5lon«l Fields ^ Corresp. Onia idtet 



Nutrition ^ 
(G) MDLINF. 43; tlOSIS, 42; A(;«MJ5, 41«; 

AC*ia>M, 4lb; AGRIRFS, 4!r; >!i:n\C»V. 45o 
S«e tUo; Food Tochnolofy. Modical Sclencef 

0 



Ocfanof raphy 

(C) OO-AX, 33*; AQUA 55b 
(G) NTIS. 71 

See also: F.urth and S^«c« Sci«nc«t 
Oil 

See: P«trol«ia 
Optics 

(G) FHYS. 12 

Sea ftlfo: Physics 
Ordnanc« 

(G) NTIS. 71; MARKETDOo/^5d 

See «l5o: Enfineerinf v 



P«p«r 

(C) PAPER. 26p 

See also: Chesistry. Forestry 
..Patents 

-(C) PATOER. 25e; PATIFI, Kb 

(S) PATCHEM77, 26a; PATCHEMSO. 2«b; 
PATtHE)CA» 26c 

See also: Dictionaries^ p. 7 
Pathology 

(G) MIOSIS, 42 J 

See also: ^Mical Sciences * 
Pedagofy 

See: Ciiucatlon 
Peti^leua 

(9^ 4)IL 57a; OJtlMCWS. STw 

St« Viso: Energy 
Pharmacoiogy * * , ^ 

(G) MEDLINE, 45; TOX, 44; TOXEFF. 444 

(S) DKUCNEMS, 4rw 

Set #Uo: Ntdical Sciances 
Philosophy 

(G)c SdCSCI, 51; HIST, S7a; A>t{IST, S7b 
Physical Scitncas 

(G) SCI, 11 

fee alto: 0i««istry» Earth 4 9pac« Sciencis> 
Englnoaring, Cenetal* Pkytics 



Physics 

(C) PHYS, 12 

(G) SCI, M; NTIS. 7\- EKISIS. 47; GfORlF. 51 

See also: Chemistry. Fugineering 
Physiology 

(G) MEDLINE, 45; ilOSJS. 42 

See als(V Life Sciences 
Phytopathology 

(G) fefOsiS, 42; AGFIUS. 41a; AGRICOM. 4lb . 

See also: Agriculture 
Plants 

(G) ilOSlS, 42; AG|tIUS, 4U; AGRICOM. 41b; 

AGRIRFS. 41r; FOOD. 41c 
See al^o: !,if« Sciences 
Political Science 
(C) PUiLIC 54c 

(G) SOCSCI, SI; HIST. 57a; AWIST, 57b 
See also: U.S. Govemsitnt 
Pollution 

(C) POLLOT, 5Sk 
(S) AIRPOLUrr, 5Sb 

See also: Environnef(t«l Science. Noise Pollution, 

Water Pollution 
Population ' 

(G) STATUS, 64b; STATUST!«, 64c; 5TATREG, 64d 

See alsb: Demography 
Power Engineering 

(C) ENERGY, 56; OfLNEMS. 57y 

(G) EB, 22 

See also: Enargy 
Printing and Publishing 

(C) LANG. S5a; LIftlNFO, SSb 

See also: Engineering 
Propulsion Systems 

Saa: Aeronautics, Navel Technology, Engineering 
Psychiatry 

(G) »CDLINE, 45; PSYCH. 55; SCI. 11; SOCSCI. 51 
Sea also! Medical Sciences 
Psychology 

(C) PSVpi, S5 4 

(G) ITOLINE, 45s SOCSfl. 51; SCI, 11; ItlOSIS, 42; 
KTI5, 71. LANG. 55A 
Public Affairs 
(C) PtItLIC, S4c 
Saa also: Political Sclance 

C • 4ata bascf that ara ccptyft i to the discipline ' 
G • data bAses that cov«r the disciplina 4 others^ to<^ 
S • data bjssos for a specific subdisciplino 



Tioff nsUm.il Field"* ^ i (>vrr*j>. |).*t i 



Q 
R 

(G) ^a:OLISE, U5; BiosTs, 42 
»lto: Life Scienctt 
Reflnlni 

R«sejirch und D*v«lop««nt 

(C) MTIS, 71; SSIR, 75 

(G) SOCSCl, 51 ^ 

(S) AGRIRES, 41 r; CANCERRES, 46t 
Robotics 

(C) ENCIN, 21; EE, 22; HB, 23 

ftlto: Artificial Tnt«llit#nc« 



5cUnc« !■ . 

(C) SCI, 11 

Alto: ^Llf« Scloncet, fhytical Scienc««« Social 

!$ci*nc»t, G«n«ral 
Social Scl«nc«t 

(C) SOCSCI, SI 

S%9 also: ficononlcs, E<hication, Library and Infor- 
•atlon S«rvlc«t, Llnfu^itlct, 'Political Sclvr^*, 
ft/cholo|y. Sociology 
Sociology 
(C) SOC, $2 

' (G) PUBLIC. $4c; S0C5C!, $lj MIST, $7a, AWIST, 57b 
Soil Sci#nc« 

^ (C) AGRIUS, 4U; ACHICOM, 41b; AGRIRES. 41r 

if alto; ^rtb and Spaca SciiMic,«t 
Spaca Scicn€«t 

XG) B!OSIS» 42; WT«OR. 12; HlYd. U 

S«a alto: Earth and SpAc« Sciim^«t 
Spac« Technology ' 

(G) ENGIN. 21; HTIS. h, SCI. 11 ^ 

5«« altot Barth and Spac« 8ci^«f»c«t, S|^a Scitncat 
Statittict 

(C) SCI, 11; Wfl* " ' * 

($> STATUS^ Mb;^ STATVStllta; 64c ;^ STATtSG* ^1 

JfrATFRN. Mf ; STAtFRHTlHi, .64g; STATnAMT,\l» 



SywMot It 

(G) BIOSIS, 42 

See also: • tlf« Sclancat 



T 



Techiwlogy"" 

S«fi» : Eng Inaerlng 
Taxtllas 

(0) RNCIN, 21; m, 23 

Set also: Chtwlstry 
Toxicology 

(C) TOX. 44, TOXEFF. 44d 

(G) AlRPOLmr, 35b 

5a« klso: Hvdlcal Scl*ncas 
Trada and CoMarca 

S«a; Busln«st 
Transportation Systa«t 

(C) ENGIN. 21; NTIS, 71; MB, 23 

Saa also: Motor V«hlclas, Aaronautics^ 
Nftval Tachnology 



u 



.Urban Planning 

(G) NTIS, 71; SOCSCf, $1; AWIST, $7*j 

AGRIUS, 41a; AGRICON, 4lb; EMVlR, 34 ^ 
iSaa i^lso: Political Scianca 
U.S, Congratf 

(C) CRECORO, S4b 
Saa alto; U.S. GovatfWMit 
U.S. Govamjaant . , 

(C) FED, S4a; FROPIOW. 54n 
(G) /A>H!ST, S7b; STAtUS, 64b; KTI8, 71; 

PUBLIC, $4^, 62 
(S) CRECQRD, S4b 

also: Political Sclanca 
U.S, History 

(C) AMIIST, 57b 
U.S. tndattrial Flmt 
(C) STATPLAMT, 64a 
Saa altp: iutinatt 
U.S. Statittict 

' (C) STATUS, 64b; STATUSTIMB, 64c; STATRgG, 64d 

C - dBtn b«t«t that Vra central t6 tha ditciplina 

G » data ^4tet that coy or ' t\\% d i t clp ll 4 othart^ too 

$ • d5ta basat for a tp^c^flc tubditciptlnBi 
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rrofonslon.i! Melds Q Corrcsp^ Ovitn Bas^s 

. V 

Veterinary Sciences 

(G) AGRIUS. 41a; AGRICOM, 41b; AGRIRF.S, 41 r; 
BIOSIS, 42 

See also: Agriculture, Life Sciences 
Virology 

(G) MEDLINE, 45 

See also: Life Sciences 



w 



hater Pollution 

(G) POLLin', 35a; OCEAN. 33a; ENVIR, 34; 
NTI-S. 71, AGRIUS, 41a; AQUA, 53b 

Sec alsot Envlronr^cntal Science 
Kildlife 

(G) BIOSIS, 42; ENVIR, 34 

See also: Aniisals, Fish, Life Sciences 



X 

n» Y 

Z 



Spec I J l Data Bases 

Contracts and Grants 

(C) FDNGRANTS, 65c; GRANTS, 65a; MARKETDQO, f^^d 

See alto: Business. 
Dictionaries : 

BiolOfical Terms - BIOTERM, 42t 

Chenical Names and Structures - CHBfTF.RM, 13t 

Qienical Toras - ^KOCHDlrrERM. 43s 

Medical Subject Headints - HEOICRH, 45t 

>ledical T«nBS - ^fE(K:llEKTERM, 45s 

Patent Classification Codes - PATTERN, 2$t 

T9&Uoloj[ical Terras - TOXTERM, 44t 

Foundations, Charitable 
(C) FONOIR, 6Sb 



Zoolofx «^ • 

See: Animal Sconce 



C • data bates that are central to tha ditciplina 
Q • data bates that covar the ditciplina I othart too 
S « data bates for a spacifie' subditeiplina 
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Attachnent 3 / 
DATA BASE DESCRIPTfONS ^ 

GiVm below «r« descriptions of the data bases Accessible through CONIT, the 
experimental interface to retrieval systems. The data bases are listed in order 
of the niuaber by which they are referred to in CONIT. 

The format of the listings is shown in the following example: 



CONIT 
No. 



CONIT 
Nana 





Ye»rs of! 


Slxe^ of^ 


^ 2 ~" 
Size and 




Usage 


jCost^ p#r 




coverage 




data base 


frequency 




cost/hr 


jrecord 


1 






as of ! 


of update 




of 




of off- 








Sept, '77 




» 


connect 




line / 














time 




printout 























Code letter of 
system having, 
the data base;' 
data base naave 
(and np.) as 
given in the 
system. 

11. SCI [D!SCISEARCH(34)] 1974-*; 1,700,000 + 42, 000/mo; $70/hr 4^ |0.10/rec. 
(SHORT DESCRIPTION OF DATA BASE) 



1 Code Letters: D » DIALOG; N - NLM/^CDLINE; 0 - ORBIT; S ■ SUNY/MEDLINE 

2 i^proximate number of documents; a data bii^c that is no longer being updated 
is tagged "FIXBD'' after its size listing. 

5. Cost is for getting aljl information in one document's computer record printed 
offline and mailed to user; partial information may cost somewhat less. 

^ 

* No final date means coverage to present date. 

NOtE: If' the infornation is different for"* different systems, then it is given 
in the order In t^ich the systens are listed: 



10. PHYSICAL Sciences (physics, chemistry) , 

11. SCI ID:SCISBARCH(54)1* 1974-; 1,700,000 + 42, 000/«o; $70/hr ♦ $.10/rec. 

Covers all fields of physical, biological, and medical literature. Corres- 
ponds to Science Citation Index published by Institute for Scientific Information. 
Articles citing a given author or paper can be retrieved. 

12- PHYS [D:lNSPEC-rPHYSICS(12)l 1969- ; 500,000 * 7,000/mo; $i'S/hTj $.10/rec. 

Covers the field of physics! Corresponds to Science A bstracts - A published 
by thjL IBB (Institute of Electrical Engineers, London) . 

13a. CHEM77 ID;CA C0NDBNSATES(4)1 1977-; 240,000 ♦ l2,000/2«k; $45/hr ♦ $.16/rec., 
Covers tha fields of chemistry and chemical engineering. Corresponds to 
Ch emical Abstracts Indexes (no abstracts) published by Chemlxral Abstracts Ser- 
vlces. Also inJexea by CASIA terms (see data base 15i). 

13b. <;HEM72ID:CA CONDBNSATES(S)) 1972-76; 1,772,194 (FIXED); $5S/hr ♦ $.08/rec. 
40:CHIil«0Nl 1972-; i,900,00OflVH)0/i wk; S60/hr ♦ $.12/vec. 
Same coverage as CHEM77 without CASIA terms. 
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Attachmont 4 

ALHIABI-TICAL LIST OF DATA llASf S,. BY CONN NAMl f, NMMB! I< " 



N.ime 



ACCOUNTANT 

AGRIUS 

AGRICOM 

AGRIRi:S 

AIRJ'OI.LUT 

A>«ili5t 

ART 

BIOSIS 

BIOTF.RM 

BUS 

CANCf R 

CANCKRRES 

CHEM7 7 

CHEM72 

CHEM70 

CHEMIN 

CHEMNEWS 

CHEMTERM 

CHILDAB 

CReCORD 

DRUGNEKS 

ED 

EOAV 

EDExani.D 

EE 

ENERGY 
ENGIN 
ENVIR 
EPIL 

FDNDIR 

FDNGRANTS 

FED 

FEDNOW 
FOOD 

CEOlltF 
GRANTS ' 



HIST 

LVNG 
LIBCONl-. 
LIBCaSF 
LIBINFO 



Number 



62b 
41a 
41b 
41r 

55b 
24b 
57b 
33b 
58 

42 

42t 
61 

46 

47r 
13a 
15b 
15c 
151 
66a 
15t 
5 2b 
54b 

66b 

56a 

56c 

56b 

22 

56 

21 

34 

4S 

65b 
65c 
S48 
54n 
41c 

31 
6Sa 



57a 

S5a 

72 c 
72f 
SSb 



MANACF- 

MAR^ETAB 

MA^ETDOO 

MARliETIN 

MAR^ETSOW 

m 

MLOAV 

^lEoa^E^^•ERM. 

MEDLINE 

MEONKiN 

MEDNOW 

MEOTEIIM 

METALS 

MhTEOR 

NT IS 

OCEA.V 
OIL 

OILNEKS 



PAPER 

PATCHEM50 

PATCHtvr? 

PATCHEMCA 

PATDER 

PATIFI 

PATTERM 

PHYS 

POLLirr 

PSYCH 
PUBLIC 

SCI 

see 

SOC?Cl 
SSIE 
STATFRN 
'STATFR\TIMh 

ST/nrn \\i 

STATRLC 
STATUS 
STATUS! i \C 

THESES 
TOX 
TQXEFF 
TOXTERM 



Nuin})c r 

6 2 a 

65a 

65d 

63i - 

63r 

25 

4.>v 

43s 

45 

45m 

43n 

43t 

24a 

52 

71 

55 

3"a 
5"v 



26p 

26b 

26a 

26c 

25a 

25b 

25t 

12 

3S8 

55 

54 c 

11 

52 

51 

73 

64 f 

64j: 

64a 

64d 

6 IS 

64c 

74 
44 
44d 
44 1 



>0 



ECU 



. 56d 



APPI-NDIX I) 

DETAILS OF 12XPER IMKNTAL SESSIONS 

In this appendix, Summarize some details of the experiinental 
: sessions which are pertinent to the analysis given in the body of the 
report* In this summary we en^hasize the problems encountered by the EU's 
in handling system mechaiiics. Search strategy considerations ^are discussed 
in Section 3. . 
EUl 

1. EUl came to the session with a written list of 11 terms from the 
ERIC thesai^rus which she had found in browsing through a printed copy of the 
thesaurus • 

2. ' EUl was confused by the message CONIT gave concerning delay in host 

system response to several search commands. Rather than have CC^IT wait for 

s ■ 

the response, she went on to give other commands, and hence lost the results 
of those searches. After seeing this happen twice, the supervisor suggested 
to the EU that she wait* in those situations; the EU followed this advice sub- 
• sequehtly in the session. After ^he session, that CONIT message was, revised 
and caused no trouble for the remaining fiVe EU's. 

■ 3. EUl asked how to request offline output. The supervisor asked her 
to try t6 get the\?cplanation through CONIT- The EU did find the correS 
explanation herself, but required four EXPLAIN commands to do it. This dif- 
ficulty related, in part„ to a general difficulty experienced by severalM^^f 
the EU's: the probl^ of finding previous explanations in a mass of typescript 

4. EUl had some trouble keeping track of the s^t numbers for the differ- 

■ I . ' "■ . ' ' 
. ent retrieved sets. At one point, she used a wrong set number in a COMBINE 
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i 



command and this hindered her search strategy formulation somewhat. ^ 

5. RUl looked at only standard (citation) output for the document 
while she was online. This was one factor in misleading her into believing 
she had retrieved documents of higher relevance to her topic than was 
act£(allyXtte case in scmo instances, 

6. EUl stopped the online session after a relatively short period 

of time (50 minutes) because: (1) she was very happy to have retrieved a 

number of good d€)cument references as a start toward a research paper; and 

(2) she ''didn't want to use too much computer time" (a constraint we had not 

f ^ 

intended; see bribfing in Appendix C) . 

V 

1. EU2 made many (eight) spelling errors caused by poor typing. As 
with other EU*s who, made relatively few errors of this type, he w^s able to 
recover from these errors fairly quickly, 

2. EU2 made a number of syntactical errors. Because he was not able 

to understand fully several features of the system, he could not always give. 

a clear explanation , in the debriefilVg, of what he was ^ trying to do; 

however, an attempt at such explanations is given below: 

a) He gave "FIND*' instead of "E FIND" when seeking an explanation • 

" of the FIND connnand.) - / 

\t) He tried to use COMBINE to "save sets". 

c) He used "SHOW WINGS" to try to get output from the "WINGS" 

search* (He did correct this in his next cojranand.) 

* . ' . ' ■ . ■• ^' 

d) He tried to combine more than two sejts at a time. 

e) He typed "SHOW ALL D3 d" in order to get "type 3 information 

. [??? Maybe Set 3], CONIT took this to mean SHOW ALL for just 
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V 

document 3 in the current set (which was set 3); perh^^ps, 
thereby, fortuitously doing what r:U2 wanted, 
f) He had trouble figuring out how to get offline output . I Because 
c the. hour was getting late, the supervisor showed the El| how this 

was done. 

" 3. EU2 preferred the unabbreviated forms of commands as easier to 
understand when reviewing what he had done. 

4. Despite all the problems, EU2 did do all the searching and online 
output without human assi^*?tance and was highly pleased and satisfied with th 
few relevant documents thus found. 

EU3 ' V 

1, Thfe retrieval system automatically logged out due to. excessive 
time without interaction while EU3 was getting the search explanations. 
CONIT's message was: "The phone connection and retrieval system have been' 
disconnected." The following dialogue ensue3%between the EU and the super- 
visor: 

EU: "Did the system crash?" 

SUP: "Yes, what do you think you should do to g^ it back?" 
EU: "Pick it again". 




SUP . "Good!" . 

I 

EU3 then picked the datfbl^e .again and was reconnected without additional 
ptoblems. 

.2. The "TIME OVERFLOW" message was received by CONIT from ORBIT 
while waiting for, the -response to a search. Due to a bug in the C(»JIT rules 



4 
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(since fixed), this caused CONIT to hang up, lathei than send ORBIT the 
request to cdhtinue searching. The superVi^or forced CONIT to do what it 
was supposed to have done, and then returned control to the 

3. This KU (3) made the error several times of omitring the space 
after the command name. He easily understood the problem and\was able t6 
continue quickly with the^ correct format. 

4. The supervisor suggested termination of the online session- after 
110 minutes for reasons of time, and because RU3 was having some difficulty 

understanding the indexing in this database (see discussion in text). 
The supervisor permitted the EU to- exceed the nominal limit of 200 offline 
references in orlier to allow the session to be completed without losing the 
current results. 

5. 'The EU was very pleased with his use of CONIT: he estimated he 
had saved about t^fSTweeks of researching in /the library. 

m . ' . ■ i 

1. EU4, as did other EU^s, complained about the dif fixity of 
looking back in the (lengthy) typescript to refer to previously printed 
explanations. ...J . 

2. The first attempt to picK the pAyS (INSPEC) database foundered 

on the attempt to connect to DIALOG over TYMNET. EU4 was somewhat con- 

.1 . ^ 

fused by th6 CONIT message suggesting he pick another system , since he had 

picked a database oversight in bringing the new CCWIT*s messages fully 

^ - .. _ ^ . •• 1 ■ ^ ■ 

into the virtual-system mode)/ Mt^ertheless EU4 was able to overcome the 

' " ' ' ' ' 
problem by simply reissuing his PtCK request. 
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3. EU4 was, as were other EU's, left unsure of what to exi^ect 
when CONIT said it would show "theVirst Ifew" documents. (CONIT did this 
because its simplified translation at that time -- of the SHOW command 
to the "PR^NT command resulted in a A^ariable number of document references.). 

4. EU4 noted a mistake in the CONIT expiration of the COMBINE 
command (the word "and" had been used instead of "or" in one example) .' 

5. Due to a bug in its rules, CONIT asked the user if he wanted 

t 

^ to wait after handling a user BREAK request.- EU4 recovered from this con- 
fusion after a prompt from the^^ supervisor that implicitly suggested he 
answer "no" to the faulty CONIT message. 

6. EU4 was somewhat confused about how maA^ times he should wait 
V for response from a search, but this did" not prevent his continuing and 

•■-is 

getting the results. 

7. An unexpected and unexplained message from DIALOG, "Msg frorrj^ 
9050:HISPEED LINE TEST", was passed on to the EU along with tW results from 
a search, ThQ, ines€age did not ~seem to bother EU4. 

8. EU4 issued the command SHOW INDEX 1, «(jpparently \sxpecting to see 
part of a thesaurus classification. He recovered quickly ("Oh, this is an 

^ index".) and inmediately issued the senmntically correct command, "SHOW INDEX 
ELECTRON. ^ 

'. 9. Due to a logical error in a CONIT rple, several of^he EU's searches 
on the ♦referejnce tags-^ (e.g., E6) for terms found from SHOW INDEX requests 
were done incorrectly. The'^ supervisor, on ^bserving this, recommended that 
searching only on the full spelling of terms be done. EU4 followed these 
directions, but his searching was hindered. 
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10. EU4 misread the instinictions to type "SMOW MORF." to continue 

a SHOW INDEX output; he typed just "MORI-" first, and then '-'SHOW MORI-", but . 
CONIT was not prepared to handle that erroneous sequenc6. fUJ4 nvas able to 
recover by redoing the original SHOW INDl-X command, but this did slow down 
his searching. 

11. EU4 was confused by the headings "Type" and "RT" in the SHOW^ 

^ INDRX response. (These^ headings were superfluous in the given context; they 
apply only if thesaurus information is available,) 

12. KU4 was confused by the free- vocabulary component of the INSPEC 
database indexing. He wondexed why there were not more documents posted 
under these terms (as there would be if they were well indexed controlled 
terms) . ^ / 

i 

-13. EU4 asked for "E FIND BETTER" (dxplanation for improving pre- 
cision) when he probably (should have) wanted "E FIND MORE" (to improve 
recall). , 

14. The truncated search on '^electron" resulted in an overf low^res-« 
'' \ - 
ponse for which CO^IT was not prepared to help tiie user. This, failure stifled 



a potentially effective line of searching. 

15. The communications with DIALOG were then disrupted and EU4 decided 
to try a different database: SCI. 

16. EU4 tried to do an author search but was foiled by thfe variant 
author format in this database (no comma after last name). In fact, EU4 wanted 
•to do a citation search, which was not yet possible in the virtua l mode. He 
also tried i^successfully to use SHOW INDEX to see authors this does not 
work for DIALOG, for which one must search the author index.* 
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17. MULTlCvS crashed, forcing a prematujr^e termination of the session; 
KU4 was disappointed with the limited results. 

BUS 

1. EU5 misread database "55a" as "55r" and usell the wrong spoiling 
in an EXPLAIN DATA conmand. 

2. EU5, still not appreciating the mistake, asked to EXPLAIN DATA 
with the name of the database (LANG). Unfortunately, the connection to DIALOG 
required for this EXPLAIN did not work. This, perhaps, led EU5 to try to 
PICK the System DIALOG, rather than a database. He then asked for the 
explanation of the LANG database again, which he finally got. 

3. ' EU5 use'd wrong syntax to get explanation for the SCI database 
(E SCI). although he had used the proper syntax twice before. This mis- 
take may be linked to his- next mistake^in^ believing that he was connected to 
SCI viien he was still connected to the default djataWse, ERIC. Thris, in turn, 

led to his searching in ERIC for 10 minutes before /realizing the mistake. 

/ 

4. EU5 issued erroneous command E FIND E<^, but then, intuitively 
figured out that*^.t should be FIND E6. 

5. Other syntactic errors were: 

a) FIND SM^ (forgot AUTHOR parameter) 

b) COMBINE 1 AND 2 (forgot SET parameter) 

c) FIND INDEX E8 (meant FIND E8) 

6. EU5 thought SHOW ALL would show all information on all documents. 

7. EU5, was happy to have retrieved the documents he did, but would 
have liked more complete recall. 
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EU6 

1. EU6, as did several other BU's, would have liked a printed 
reference manual to refer to. 

2. EU6, as was ^rue of several other EU's, did not always wait for 
the USER cue. These mishaps caused minor delays, but generally were easily 
understood and overcome. 

X. EU6 searched on several problems; this prevented more in-d^pth 
searching of any one problem. 

4. Late in the session, EU6 was unsure of the proper use (Jf 
CONffllNE, even though he had used it properly earlier. The incomplete form, 
COMBINE SET, led^ to the suggestion to E COMBINE, which in turn reminded EU6 
of the correct usage. 

5. EU6,was happy to have retrieved the, document s^e did, but would 
have preferred more complete recall. 



